-- Logs begin at Thu 2022-07-07 14:32:42 UTC, end at Thu 2022-07-07 15:50:48 UTC. -- Jul 07 15:14:00.223407 dv-cinder-841302-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 07 15:14:00.232084 dv-cinder-841302-5-iscsi devstack@keystone.service[80289]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 07 15:14:00.239310 dv-cinder-841302-5-iscsi devstack@keystone.service[80289]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 7 15:14:00 2022] *** Jul 07 15:14:00.239310 dv-cinder-841302-5-iscsi devstack@keystone.service[80289]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 07 15:14:00.239310 dv-cinder-841302-5-iscsi devstack@keystone.service[80289]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 07 15:14:00.239310 dv-cinder-841302-5-iscsi devstack@keystone.service[80289]: nodename: dv-cinder-841302-5-iscsi Jul 07 15:14:00.239310 dv-cinder-841302-5-iscsi devstack@keystone.service[80289]: machine: x86_64 Jul 07 15:14:00.239310 dv-cinder-841302-5-iscsi devstack@keystone.service[80289]: clock source: unix Jul 07 15:14:00.239310 dv-cinder-841302-5-iscsi devstack@keystone.service[80289]: pcre jit disabled Jul 07 15:14:00.239310 dv-cinder-841302-5-iscsi devstack@keystone.service[80289]: detected number of CPU cores: 4 Jul 07 15:14:00.239310 dv-cinder-841302-5-iscsi devstack@keystone.service[80289]: current working directory: / Jul 07 15:14:00.239310 dv-cinder-841302-5-iscsi devstack@keystone.service[80289]: detected binary path: /usr/bin/uwsgi-core Jul 07 15:14:00.239310 dv-cinder-841302-5-iscsi devstack@keystone.service[80289]: your processes number limit is 47680 Jul 07 15:14:00.239310 dv-cinder-841302-5-iscsi devstack@keystone.service[80289]: your memory page size is 4096 bytes Jul 07 15:14:00.239310 dv-cinder-841302-5-iscsi devstack@keystone.service[80289]: detected max file descriptor number: 2048 Jul 07 15:14:00.239310 dv-cinder-841302-5-iscsi devstack@keystone.service[80289]: lock engine: pthread robust mutexes Jul 07 15:14:00.239310 dv-cinder-841302-5-iscsi devstack@keystone.service[80289]: thunder lock: enabled Jul 07 15:14:00.239310 dv-cinder-841302-5-iscsi devstack@keystone.service[80289]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 07 15:14:00.239310 dv-cinder-841302-5-iscsi devstack@keystone.service[80289]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 07 15:14:00.277830 dv-cinder-841302-5-iscsi devstack@keystone.service[80289]: Python main interpreter initialized at 0x55bdac17b9c0 Jul 07 15:14:00.277830 dv-cinder-841302-5-iscsi devstack@keystone.service[80289]: python threads support enabled Jul 07 15:14:00.277830 dv-cinder-841302-5-iscsi devstack@keystone.service[80289]: your server socket listen backlog is limited to 100 connections Jul 07 15:14:00.277830 dv-cinder-841302-5-iscsi devstack@keystone.service[80289]: your mercy for graceful operations on workers is 90 seconds Jul 07 15:14:00.282728 dv-cinder-841302-5-iscsi devstack@keystone.service[80289]: mapped 403077 bytes (393 KB) for 2 cores Jul 07 15:14:00.282728 dv-cinder-841302-5-iscsi devstack@keystone.service[80289]: *** Operational MODE: preforking *** Jul 07 15:14:00.282728 dv-cinder-841302-5-iscsi devstack@keystone.service[80289]: *** uWSGI is running in multiple interpreter mode *** Jul 07 15:14:00.282728 dv-cinder-841302-5-iscsi devstack@keystone.service[80289]: spawned uWSGI master process (pid: 80289) Jul 07 15:14:00.282728 dv-cinder-841302-5-iscsi devstack@keystone.service[80289]: spawned uWSGI worker 1 (pid: 80300, cores: 1) Jul 07 15:14:00.282728 dv-cinder-841302-5-iscsi devstack@keystone.service[80289]: spawned uWSGI worker 2 (pid: 80301, cores: 1) Jul 07 15:14:00.282728 dv-cinder-841302-5-iscsi devstack@keystone.service[80289]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 07 15:14:00.278703 dv-cinder-841302-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 07 15:14:05.761047 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 07 15:14:05.761047 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: from cryptography.utils import int_from_bytes Jul 07 15:14:05.761850 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 07 15:14:05.761850 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: from cryptography.utils import int_from_bytes Jul 07 15:14:05.761964 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 07 15:14:05.761964 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: from cryptography.utils import int_from_bytes Jul 07 15:14:05.761964 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 07 15:14:05.761964 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: from cryptography.utils import int_from_bytes Jul 07 15:14:05.811650 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 07 15:14:05.811811 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 07 15:14:05.811906 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 07 15:14:05.811993 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 07 15:14:05.812067 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] command line args: [] {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 07 15:14:05.812142 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] command line args: [] {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 07 15:14:05.812300 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 07 15:14:05.812387 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 07 15:14:05.812465 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ================================================================================ {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 07 15:14:05.812540 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ================================================================================ {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 07 15:14:05.812712 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] admin_token = **** {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.812795 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] admin_token = **** {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.812955 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] config_dir = [] {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.813038 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] config_dir = [] {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.813140 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.813222 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.813322 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] config_source = [] {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.813403 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] config_source = [] {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.813481 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] debug = True {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.813553 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] debug = True {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.813681 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.813946 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.814041 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.814106 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.814259 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.814340 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.814456 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.814575 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.814699 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.814817 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.814934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] list_limit = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.815044 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] list_limit = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.815159 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] log_config_append = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.815275 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] log_config_append = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.815397 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.815514 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.815667 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] log_dir = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.815782 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] log_dir = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.815899 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] log_file = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.816015 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] log_file = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.816137 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.816259 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.816376 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.816493 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.816595 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.816688 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.816804 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.816927 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.817036 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.817139 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.817255 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.817378 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.817459 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.817567 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.817637 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.817717 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.817793 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.817915 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.817989 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.818106 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.818180 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.818296 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.818370 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.818491 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.818566 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.818689 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.818763 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.818881 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.818960 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.819041 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.819400 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.819480 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.819683 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] publish_errors = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.819763 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] publish_errors = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.819878 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.819952 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.820064 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.820145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.820215 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.820281 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.820437 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.820530 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.820670 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.820746 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.820903 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] standard_threads = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.820986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] standard_threads = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.821134 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.821224 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.821377 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.821456 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.821532 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.821651 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.821726 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] use_journal = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.821806 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] use_journal = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.821877 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] use_json = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.821990 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] use_json = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.822064 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] use_stderr = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.822134 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] use_stderr = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.822209 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] use_syslog = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.822355 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] use_syslog = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.822513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.822598 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 15:14:05.822669 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.822744 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.822870 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.822956 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.823129 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.823215 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.823369 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.823447 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.823640 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.823731 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.823883 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.823959 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.824130 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.824213 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.824367 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.824450 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.824612 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.824688 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.824858 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.824933 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.825104 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.825187 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.825337 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.825412 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.825580 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.825661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.825816 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.825928 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.826062 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.826143 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.826306 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.826389 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.826542 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.826625 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.826795 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.826871 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.827055 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.827132 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.827300 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.827378 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.827535 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.827623 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.827814 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.827889 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.828077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.828161 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.828330 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.828407 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.828578 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.828654 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.828809 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.828888 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.829041 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.829117 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.829270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.829347 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.829507 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.829584 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.829752 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.829829 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.830001 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.830077 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.830231 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.830315 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.830461 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.830537 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.830709 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.830785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.830955 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.831037 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.831191 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.831268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.831461 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.831538 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.831717 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.831796 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.831952 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.832030 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.832171 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.832287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.832398 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.832498 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.832620 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.832728 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.832850 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.832958 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.833085 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.833195 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.833326 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.833407 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.833578 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.833669 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.833813 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.833892 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.834053 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.834127 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.834288 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.834365 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.834524 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.834607 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.834761 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.834841 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.835007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.835079 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.835239 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.835323 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.835478 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.835555 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.835737 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.835814 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.835974 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.836061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.836211 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.836287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.836449 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.836526 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.836682 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.836766 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.836939 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.837023 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.837178 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.837251 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.837412 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.837489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.837644 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.837721 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.837903 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.838040 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.838142 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.838226 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.838375 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.838451 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.838613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.838695 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.838846 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.838933 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.839113 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.839194 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.839347 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.839423 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.839616 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.839776 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.839858 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.839929 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.840088 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.840166 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.840323 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.840408 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.840558 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.840634 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.840804 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.840880 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.841041 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.841132 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.841276 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.841358 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.841516 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.841592 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.841753 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.841864 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.841986 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.842068 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.842219 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.842295 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.842450 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.842526 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.842684 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.842760 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.842942 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.843018 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.843181 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.843259 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.843418 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.843495 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.843676 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.843754 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.843918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.843994 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.844155 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.844239 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.844392 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.844469 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.844626 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.844702 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.844855 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.844933 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.845084 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.845160 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.845313 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.845395 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.845547 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.845629 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.845780 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.845856 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.846018 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.846095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.846254 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.846336 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.846499 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.846575 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.846735 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.846810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.847772 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.847897 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.848047 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.848136 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.848287 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.848363 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.848518 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] auth.external = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.848595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] auth.external = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.848751 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.848828 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.849052 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.849164 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.849285 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.849364 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.849515 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] auth.password = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.849592 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] auth.password = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.849743 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] auth.token = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.849825 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] auth.token = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.849981 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.850057 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.850215 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.850335 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.850460 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.850537 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.850687 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.850764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.850934 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.851018 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.851177 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.851256 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.851406 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.851482 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.851662 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] credential.caching = True {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.851743 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] credential.caching = True {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.851910 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.851987 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.852156 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.852234 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.852403 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.852489 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.852637 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.852714 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.852878 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.852962 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.853110 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.853188 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.853369 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.853447 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.853620 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.853696 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.853856 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.853940 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.854106 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.854184 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.854387 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.854543 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.854620 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.854691 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.854878 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.854956 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.855117 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.855201 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.855370 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.855448 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.855623 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] federation.caching = True {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.855702 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] federation.caching = True {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.855860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.855949 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.856107 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.856186 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.856354 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.856432 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.856581 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.856667 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.856826 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.856904 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.857058 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.857135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.857310 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.857394 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.857548 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.857625 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.857801 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.857878 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.858032 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.858117 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.858277 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.858359 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.858511 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] identity.caching = True {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.858588 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] identity.caching = True {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.858757 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.858842 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.859003 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.859081 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.859239 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.859315 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.859472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.859558 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.859740 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.859829 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.859969 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.860047 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.860207 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.860291 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.860453 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.860530 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.860701 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.860785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.860930 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.861016 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.861159 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.861236 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.861389 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.861465 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.861632 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.861717 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.861876 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.861954 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.862126 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.862202 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.862380 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.862465 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.862629 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.863313 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.863465 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.863547 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.863730 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.863824 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.863970 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.864048 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.864203 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.864281 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.864439 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.864518 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.864670 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.864754 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.864905 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.864988 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.865145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.865225 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.865387 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.865464 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.865635 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.865712 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.865867 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.865952 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.866114 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.866190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.866359 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.866436 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.866595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.866672 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.866841 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.866919 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.867087 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.867171 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.867317 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.867395 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.867558 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.867651 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.867810 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.867889 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.868043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.868122 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.868278 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.868355 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.868517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.868595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.868753 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.868836 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.868988 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.869066 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.869235 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.869312 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.869481 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.869560 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.869710 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.869786 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.869937 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.870014 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.870181 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.870260 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.870426 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.870503 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.870661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.870737 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.870894 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.870972 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.871129 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.871213 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.871359 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.user = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.871436 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.user = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.871615 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.871698 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.871878 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.871955 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.872108 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.872185 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.872355 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.872434 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.872598 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.872675 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.872844 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.872921 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.873078 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.873157 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.873306 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.873382 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.873538 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.873615 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.873772 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.873850 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.874003 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.874080 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.874232 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.874309 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.874476 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.874555 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.874722 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.874804 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.874964 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.875041 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.875209 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.875287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.875452 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.875529 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.875703 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.875780 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.875949 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.876028 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.876185 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.876262 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.876415 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.876493 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.876649 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.876727 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.876880 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.876958 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.877123 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.877201 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.877371 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.877450 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.877605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.877682 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.877858 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.877934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.878077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.878151 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.878335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.878414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.878588 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.878672 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.878806 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.878885 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.879040 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.879117 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.879284 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.879367 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.879524 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.879656 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.879779 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.879857 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.880011 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.880101 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.880252 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] resource.caching = True {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.880329 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] resource.caching = True {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.880498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.880577 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.880745 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.880830 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.880977 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.881054 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.881224 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.881300 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.881468 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.881553 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.881705 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.881784 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.881950 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.882027 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.882181 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.882266 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.882419 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.882496 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.882652 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] role.caching = True {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.882730 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] role.caching = True {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.882897 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] role.driver = sql {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.882983 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] role.driver = sql {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.883127 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.883204 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.883369 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.883446 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.883639 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.883733 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.883890 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.883969 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.884137 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.884217 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.884382 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.884469 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.884627 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.884713 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.884871 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.884949 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.885117 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.885202 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.885346 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.885423 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.885592 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.885669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.885838 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.885923 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.886082 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.886159 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.886327 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.886404 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.886713 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.886798 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.886947 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.887024 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.887192 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.887270 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.887444 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.887522 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.887716 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.887796 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.887965 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.888041 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.888197 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.888280 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.888450 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.888527 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.888702 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.888779 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.888938 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.889017 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.889167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.889244 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.889398 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.889476 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.889628 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.889707 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.889877 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.889954 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.890131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.890207 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.890374 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.890452 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.890611 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.890689 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.890845 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.890921 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.891073 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.891151 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.891308 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] token.caching = True {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.891386 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] token.caching = True {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.891542 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.891676 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.891813 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.891893 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.892049 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.892128 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.892303 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.892380 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.892549 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.892627 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.892784 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.892860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.893024 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.893100 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.893265 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.893344 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.893510 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.893586 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.893741 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.893818 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.894003 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.894081 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.894236 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.894313 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.894481 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.894557 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.894725 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.894803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.894953 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.895030 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.895193 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.895270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.895482 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.895609 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.895764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] database.connection = **** {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.895854 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] database.connection = **** {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.896006 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.896092 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.896251 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.896331 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.896486 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.896573 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.896724 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.896802 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.896957 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.897036 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.897191 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.897342 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.897418 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.897489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.897648 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.897727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.897888 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.897973 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.898120 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.898197 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.898361 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.898442 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.898592 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.898681 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.898836 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.898913 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.899068 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.899145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.899307 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.899391 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.899533 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.899629 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.899795 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.899872 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.900027 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.900116 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 15:14:05.900191 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 07 15:14:05.900262 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80301) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 07 15:14:05.931740 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80301) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:05.931894 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:05.931979 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80301) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:05.932110 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:05.932192 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80301) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:05.932346 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:05.940686 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80301) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:05.940790 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:05.969186 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80301) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:05.969421 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80301) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:05.969504 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:05.969737 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.112046 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80301) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.112310 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.160584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80301) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.160817 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80301) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.161710 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.161953 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.226517 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.226725 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80301) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.226820 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.226956 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.227098 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80301) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.227185 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.227262 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80301) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.227378 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.227461 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80301) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.227614 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80301) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.227700 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.227836 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80301) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.251452 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.251711 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.251851 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80301) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.251953 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.252085 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80301) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.252168 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.252313 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80301) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.252395 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.252538 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80301) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.252621 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.252756 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.252837 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80301) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.252970 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80301) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.253186 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80301) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.263869 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80301) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.263966 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.277179 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80301) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.277391 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.277500 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80301) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.277580 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.277654 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80301) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.277761 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.277844 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80301) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.277979 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.278070 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80301) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.278193 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.278276 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80301) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.278406 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80301) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.278487 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.278616 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 15:14:06.296268 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.296477 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 07 15:14:06.296814 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.296902 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.297016 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 07 15:14:06.297114 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 07 15:14:06.297288 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.297370 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.297544 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 07 15:14:06.297623 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.297800 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.297881 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.298056 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.298137 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.298314 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.298572 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.308928 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.309205 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.309476 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.309558 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.309754 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.309851 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.310095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.310376 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.318027 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80301) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 15:14:06.318683 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80300) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 15:14:06.321321 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80301) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 15:14:06.321605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.321889 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.322011 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80300) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 15:14:06.322168 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.322293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.322452 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.322578 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.322712 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.322856 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.322976 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.323136 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.323246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.323391 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.323511 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.323682 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.323807 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.323955 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.324071 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.324223 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.324492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.324758 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.351818 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.353436 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80301) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 15:14:06.353525 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.353715 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.355119 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80300) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 15:14:06.355402 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.358967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80301) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 15:14:06.359245 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.359526 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.360702 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80300) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 15:14:06.360986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.361382 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.367001 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80301) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 15:14:06.367246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.368740 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80300) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 15:14:06.369028 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.371737 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80301) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 15:14:06.372025 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.372305 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.372586 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.372863 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.373137 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.373404 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.373522 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80300) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 15:14:06.373666 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.373808 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.374092 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.374376 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.374654 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.374923 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.375188 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.375453 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.392757 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.393031 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.395334 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.395642 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.398221 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80301) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 15:14:06.400159 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80300) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 15:14:06.401878 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.402176 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.403868 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.404165 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.407314 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80301) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 15:14:06.409288 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80300) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 15:14:06.410979 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80301) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 15:14:06.412949 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80300) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 15:14:06.414628 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.414927 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.415212 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.415496 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.415814 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.416099 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.416605 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.416911 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.417203 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.417503 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.417783 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.418060 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.435015 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80301) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 15:14:06.435301 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.435603 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.435872 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.437257 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80300) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 15:14:06.437545 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.437815 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.438077 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.444294 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.445995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.446271 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.446580 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.448327 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.448607 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.449476 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80301) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 15:14:06.449759 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.450046 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.450333 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.450660 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.451837 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80300) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 15:14:06.452119 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.452398 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.452677 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.452956 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.464543 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80301) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 15:14:06.464822 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.465098 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.465383 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.465661 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.465951 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.466224 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.466828 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80300) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 15:14:06.467111 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.467393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.467705 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.467989 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.469536 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.469830 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.485568 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80301) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 15:14:06.488033 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80300) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 15:14:06.488874 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80301) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 15:14:06.491321 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80300) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 15:14:06.492202 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.493810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.494685 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.495584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80301) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 15:14:06.495873 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.496162 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.496461 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.498103 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80300) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 15:14:06.498386 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.498671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.503976 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.505584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80301) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 15:14:06.506518 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.508275 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80300) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 15:14:06.508846 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.509136 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.509420 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.509694 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.511515 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.511829 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.512225 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.512508 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.518876 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80301) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 15:14:06.519724 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.520008 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.522159 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80300) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 15:14:06.522448 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.522731 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.527855 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80301) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 15:14:06.528134 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.528414 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.528689 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.528958 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.529242 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.529514 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.529784 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.530149 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.530430 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.530581 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80300) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 15:14:06.530709 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.530862 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.530973 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.531139 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.531248 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.531408 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.531517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80301) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.531700 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.532094 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.532374 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.532662 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.532939 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.533222 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.533504 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.533770 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.534038 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.534308 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 15:14:06.569563 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55bdac17b9c0 pid: 80301 (default app) Jul 07 15:14:06.572694 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55bdac17b9c0 pid: 80300 (default app) Jul 07 15:14:06.572921 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3be5f-bcca-42e1-9fd6-36bf357b0642 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:06.573213 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3be5f-bcca-42e1-9fd6-36bf357b0642 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:06.573523 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3be5f-bcca-42e1-9fd6-36bf357b0642 None None] PATH_INFO: `/v3` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:06.577773 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1/1] 10.222.0.25 () {50 vars in 791 bytes} [Thu Jul 7 15:14:06 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:15.020017 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7d734-c796-41fb-81da-b5437fceaf56 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:15.020538 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7d734-c796-41fb-81da-b5437fceaf56 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:15.020630 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7d734-c796-41fb-81da-b5437fceaf56 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:15.024971 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1/2] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:15 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:14:15.030172 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136dcda-03eb-481b-8ad9-e61c8d93dcae None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:15.030612 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136dcda-03eb-481b-8ad9-e61c8d93dcae None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:15.030755 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136dcda-03eb-481b-8ad9-e61c8d93dcae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:15.100181 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: INFO dbcounter [None req-6136dcda-03eb-481b-8ad9-e61c8d93dcae None None] Registered counter for database keystone Jul 07 15:14:15.121364 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG oslo_db.sqlalchemy.engines [None req-6136dcda-03eb-481b-8ad9-e61c8d93dcae None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80301) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 07 15:14:15.127725 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writer thread running {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 07 15:14:15.301281 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG passlib.handlers.bcrypt [None req-6136dcda-03eb-481b-8ad9-e61c8d93dcae None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80301) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 07 15:14:15.301560 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG passlib.handlers.bcrypt [None req-6136dcda-03eb-481b-8ad9-e61c8d93dcae None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80301) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 07 15:14:15.356559 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-6136dcda-03eb-481b-8ad9-e61c8d93dcae None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:15.358150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-6136dcda-03eb-481b-8ad9-e61c8d93dcae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:15.484308 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 2/3] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:15 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 455 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 07 15:14:15.670761 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7d734-c796-41fb-81da-b5437fceaf56 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:15.745194 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: INFO dbcounter [None req-e2a7d734-c796-41fb-81da-b5437fceaf56 None None] Registered counter for database keystone Jul 07 15:14:15.764811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG oslo_db.sqlalchemy.engines [None req-e2a7d734-c796-41fb-81da-b5437fceaf56 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80300) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 07 15:14:15.770906 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writer thread running {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 07 15:14:15.887045 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7d734-c796-41fb-81da-b5437fceaf56 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:15.891258 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d80789-65e7-4a90-96c2-cedaa8417f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:15.891933 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d80789-65e7-4a90-96c2-cedaa8417f33 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:15.892216 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d80789-65e7-4a90-96c2-cedaa8417f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:15.892484 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d80789-65e7-4a90-96c2-cedaa8417f33 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:15.900477 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d80789-65e7-4a90-96c2-cedaa8417f33 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:15.951471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d9d80789-65e7-4a90-96c2-cedaa8417f33 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 07 15:14:15.951779 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d80789-65e7-4a90-96c2-cedaa8417f33 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:15.961411 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 2/4] 10.222.0.25 () {58 vars in 1324 bytes} [Thu Jul 7 15:14:15 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 07 15:14:15.967239 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136dcda-03eb-481b-8ad9-e61c8d93dcae None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:15.981715 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136dcda-03eb-481b-8ad9-e61c8d93dcae None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:15.986068 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed675822-99be-453e-aed4-be7664599690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:15.987181 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed675822-99be-453e-aed4-be7664599690 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:15.988016 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed675822-99be-453e-aed4-be7664599690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:15.988289 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed675822-99be-453e-aed4-be7664599690 admin admin] PATH_INFO: `/v3/projects/28133f9a98be4af88ed357f76f5a4a2e` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:15.992308 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed675822-99be-453e-aed4-be7664599690 admin admin] RBAC: Authorizing `identity:get_project(project_id=28133f9a98be4af88ed357f76f5a4a2e)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:16.041685 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ed675822-99be-453e-aed4-be7664599690 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 07 15:14:16.042192 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed675822-99be-453e-aed4-be7664599690 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:16.046543 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 3/5] 10.222.0.25 () {56 vars in 1207 bytes} [Thu Jul 7 15:14:15 2022] GET /identity/v3/projects/28133f9a98be4af88ed357f76f5a4a2e => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:18.010443 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9f6e5-f35a-4f56-8cc5-72647dc6c1eb None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:18.010876 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9f6e5-f35a-4f56-8cc5-72647dc6c1eb None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:18.011005 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9f6e5-f35a-4f56-8cc5-72647dc6c1eb None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:18.015180 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 3/6] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:14:18.020896 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e2982-406d-487e-9056-68af35936caa None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:18.021324 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e2982-406d-487e-9056-68af35936caa None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:18.021451 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e2982-406d-487e-9056-68af35936caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:18.040749 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-3a5e2982-406d-487e-9056-68af35936caa None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:18.042279 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-3a5e2982-406d-487e-9056-68af35936caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:18.050530 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 4/7] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:18 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 07 15:14:18.072469 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae9f6e5-f35a-4f56-8cc5-72647dc6c1eb None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:18.084871 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae9f6e5-f35a-4f56-8cc5-72647dc6c1eb None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:18.089037 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9159e43d-3a6e-4a41-8ceb-c6c372a741b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:18.089664 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159e43d-3a6e-4a41-8ceb-c6c372a741b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:18.089950 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159e43d-3a6e-4a41-8ceb-c6c372a741b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:18.090254 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159e43d-3a6e-4a41-8ceb-c6c372a741b9 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:18.098799 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9159e43d-3a6e-4a41-8ceb-c6c372a741b9 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:18.100630 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9159e43d-3a6e-4a41-8ceb-c6c372a741b9 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:18.109380 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 4/8] 10.222.0.25 () {58 vars in 1324 bytes} [Thu Jul 7 15:14:18 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 07 15:14:18.114508 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e2982-406d-487e-9056-68af35936caa None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:18.123321 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e2982-406d-487e-9056-68af35936caa None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:18.127711 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793d5d8-a336-421e-b314-d45c5b0780e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:18.128584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793d5d8-a336-421e-b314-d45c5b0780e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:18.128866 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793d5d8-a336-421e-b314-d45c5b0780e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:18.129141 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793d5d8-a336-421e-b314-d45c5b0780e4 admin admin] PATH_INFO: `/v3/users/30014035e7d641d5aecffdbe884d3d52` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:18.130513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793d5d8-a336-421e-b314-d45c5b0780e4 admin admin] RBAC: Authorizing `identity:get_user(user_id=30014035e7d641d5aecffdbe884d3d52)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:18.132360 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793d5d8-a336-421e-b314-d45c5b0780e4 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:18.133894 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 5/9] 10.222.0.25 () {56 vars in 1198 bytes} [Thu Jul 7 15:14:18 2022] GET /identity/v3/users/30014035e7d641d5aecffdbe884d3d52 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:20.182823 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bad894-99ed-4074-9770-63dab818f1ae None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:20.183352 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bad894-99ed-4074-9770-63dab818f1ae None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:20.183447 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bad894-99ed-4074-9770-63dab818f1ae None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:20.184812 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 5/10] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:14:20.189840 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1666e7d-cf80-442c-aba8-3cfe2d86ef28 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:20.190264 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1666e7d-cf80-442c-aba8-3cfe2d86ef28 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:20.190940 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1666e7d-cf80-442c-aba8-3cfe2d86ef28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:20.208133 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-c1666e7d-cf80-442c-aba8-3cfe2d86ef28 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:20.209294 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-c1666e7d-cf80-442c-aba8-3cfe2d86ef28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:20.238319 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d9c4ec-e374-4fe9-89ae-97144b9feb86 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:20.238531 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d9c4ec-e374-4fe9-89ae-97144b9feb86 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:20.238799 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d9c4ec-e374-4fe9-89ae-97144b9feb86 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:20.240148 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 6/11] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:14:20.244969 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd3447b-a396-42d9-a304-eea9ab4ac647 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:20.245252 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd3447b-a396-42d9-a304-eea9ab4ac647 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:20.245532 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd3447b-a396-42d9-a304-eea9ab4ac647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:20.289707 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 6/12] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:14:20.292183 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG passlib.handlers.bcrypt [None req-2fd3447b-a396-42d9-a304-eea9ab4ac647 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80300) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 07 15:14:20.292615 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG passlib.handlers.bcrypt [None req-2fd3447b-a396-42d9-a304-eea9ab4ac647 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80300) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 07 15:14:20.312341 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1666e7d-cf80-442c-aba8-3cfe2d86ef28 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:20.334960 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:20.338677 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-2fd3447b-a396-42d9-a304-eea9ab4ac647 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:20.340334 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-2fd3447b-a396-42d9-a304-eea9ab4ac647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:20.421269 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1666e7d-cf80-442c-aba8-3cfe2d86ef28 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:20.448058 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 7/13] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 15:14:20.455842 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80224d2-0a3b-4dc5-bd3b-274cfd213304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:20.456663 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80224d2-0a3b-4dc5-bd3b-274cfd213304 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:20.456948 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80224d2-0a3b-4dc5-bd3b-274cfd213304 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:20.457228 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80224d2-0a3b-4dc5-bd3b-274cfd213304 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:20.468327 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80224d2-0a3b-4dc5-bd3b-274cfd213304 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:14:20.469965 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd3447b-a396-42d9-a304-eea9ab4ac647 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:20.479860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-b80224d2-0a3b-4dc5-bd3b-274cfd213304 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:14:20.479860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:14:20.479860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:14:20.479860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:14:20.479860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:14:20.479860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:14:20.479860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:14:20.479860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:14:20.479860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:14:20.479860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:14:20.479860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:14:20.479860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:14:20.479860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:14:20.479860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:14:20.479860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:14:20.479860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:14:20.481003 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:14:20.481003 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:14:20.481003 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:14:20.481003 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:14:20.481003 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:14:20.481003 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:14:20.481003 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:14:20.481003 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:14:20.481003 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:14:20.481003 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:14:20.481003 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:14:20.481003 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:14:20.481003 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:14:20.481003 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:14:20.481003 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:14:20.481003 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:14:20.482031 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:14:20.482031 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:14:20.482031 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:14:20.482031 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:14:20.482031 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:14:20.482031 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:14:20.482031 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:14:20.482031 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:14:20.482031 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:14:20.482031 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:14:20.482031 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:14:20.482031 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:14:20.482031 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 7/14] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Jul 7 15:14:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:14:20.486613 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80224d2-0a3b-4dc5-bd3b-274cfd213304 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:20.570266 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:20.572321 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd3447b-a396-42d9-a304-eea9ab4ac647 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:20.574628 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:20.587744 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80224d2-0a3b-4dc5-bd3b-274cfd213304 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:20.610121 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58fb0be-b371-4d31-a9f4-a73ac7bf0a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:20.611009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58fb0be-b371-4d31-a9f4-a73ac7bf0a56 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:20.611294 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58fb0be-b371-4d31-a9f4-a73ac7bf0a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:20.611590 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58fb0be-b371-4d31-a9f4-a73ac7bf0a56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:20.623463 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621e14a-ad26-42de-8d32-779f5c1991a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:20.624153 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621e14a-ad26-42de-8d32-779f5c1991a3 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:20.624493 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621e14a-ad26-42de-8d32-779f5c1991a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:20.624771 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621e14a-ad26-42de-8d32-779f5c1991a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:20.625524 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d621e14a-ad26-42de-8d32-779f5c1991a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:20.625828 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d621e14a-ad26-42de-8d32-779f5c1991a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:20.627274 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d621e14a-ad26-42de-8d32-779f5c1991a3 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:20.638721 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 8/15] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:14:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:14:20.644116 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621e14a-ad26-42de-8d32-779f5c1991a3 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:20.679807 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58fb0be-b371-4d31-a9f4-a73ac7bf0a56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:20.681337 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58fb0be-b371-4d31-a9f4-a73ac7bf0a56 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:20.747120 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621e14a-ad26-42de-8d32-779f5c1991a3 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:20.780217 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b20572-edba-47da-acac-dfb8816c14d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:20.781085 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b20572-edba-47da-acac-dfb8816c14d8 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:20.781364 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b20572-edba-47da-acac-dfb8816c14d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:20.781636 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b20572-edba-47da-acac-dfb8816c14d8 None admin] PATH_INFO: `/v3/users/30014035e7d641d5aecffdbe884d3d52` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:20.782827 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b20572-edba-47da-acac-dfb8816c14d8 None admin] RBAC: Authorizing `identity:get_user(user_id=30014035e7d641d5aecffdbe884d3d52)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:20.784812 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b20572-edba-47da-acac-dfb8816c14d8 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:20.786422 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 9/16] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:14:20 2022] GET /identity/v3/users/30014035e7d641d5aecffdbe884d3d52 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:14:20.791863 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b20572-edba-47da-acac-dfb8816c14d8 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:20.792110 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 8/17] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:14:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 07 15:14:20.797377 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58fb0be-b371-4d31-a9f4-a73ac7bf0a56 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:20.800946 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:20.811065 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:20.893445 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b20572-edba-47da-acac-dfb8816c14d8 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:20.895663 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58fb0be-b371-4d31-a9f4-a73ac7bf0a56 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:20.929899 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f1f7a2-6a67-4000-bfdb-edf26a02e0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:20.930282 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f93c93c-4059-4b7c-865f-763d600f662b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:20.930686 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1f7a2-6a67-4000-bfdb-edf26a02e0f2 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:20.930970 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1f7a2-6a67-4000-bfdb-edf26a02e0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:20.931068 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f93c93c-4059-4b7c-865f-763d600f662b None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:20.931244 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1f7a2-6a67-4000-bfdb-edf26a02e0f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:20.931342 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f93c93c-4059-4b7c-865f-763d600f662b None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:20.931602 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f93c93c-4059-4b7c-865f-763d600f662b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:20.932448 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f1f7a2-6a67-4000-bfdb-edf26a02e0f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:20.933902 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f1f7a2-6a67-4000-bfdb-edf26a02e0f2 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:20.935499 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 10/18] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:14:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:14:20.941104 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f1f7a2-6a67-4000-bfdb-edf26a02e0f2 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:20.949678 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f93c93c-4059-4b7c-865f-763d600f662b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:20.951885 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f93c93c-4059-4b7c-865f-763d600f662b None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:20.973958 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-4f93c93c-4059-4b7c-865f-763d600f662b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:14:20.973958 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:14:20.973958 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:14:20.973958 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:14:20.973958 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:14:20.973958 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:14:20.973958 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:14:20.973958 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:14:20.973958 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:14:20.973958 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:14:20.973958 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:14:20.973958 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:14:20.973958 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:14:20.973958 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:14:20.973958 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:14:20.974960 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:14:20.974960 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:14:20.974960 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:14:20.974960 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:14:20.974960 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:14:20.974960 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:14:20.974960 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:14:20.974960 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:14:20.974960 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:14:20.974960 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:14:20.974960 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:14:20.974960 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:14:20.974960 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:14:20.974960 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:14:20.974960 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:14:20.974960 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:14:20.974960 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:14:20.975997 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:14:20.975997 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:14:20.975997 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:14:20.975997 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:14:20.975997 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 9/19] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:14:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:14:20.981964 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f93c93c-4059-4b7c-865f-763d600f662b None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:21.025813 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:21.045491 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:21.046454 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f1f7a2-6a67-4000-bfdb-edf26a02e0f2 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:21.080025 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9acce5-e276-448a-9176-95a218b6975c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:21.080690 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9acce5-e276-448a-9176-95a218b6975c None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:21.081499 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9acce5-e276-448a-9176-95a218b6975c None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:21.081779 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9acce5-e276-448a-9176-95a218b6975c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:21.082614 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9acce5-e276-448a-9176-95a218b6975c None admin] RBAC: Adding query filter params (role.id=0e7793fbf3064037984da651879e3a94, scope.domain.id=default, user.id=30014035e7d641d5aecffdbe884d3d52) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:21.082927 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9acce5-e276-448a-9176-95a218b6975c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:21.084402 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9acce5-e276-448a-9176-95a218b6975c None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:21.088757 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f93c93c-4059-4b7c-865f-763d600f662b None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:21.091001 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 11/20] 10.222.0.25 () {56 vars in 1322 bytes} [Thu Jul 7 15:14:20 2022] GET /identity/v3/role_assignments?user.id=30014035e7d641d5aecffdbe884d3d52&scope.domain.id=default&role.id=0e7793fbf3064037984da651879e3a94 => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:14:21.125387 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e603a50-6f29-4d26-8678-420c0f5eac72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:21.126213 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e603a50-6f29-4d26-8678-420c0f5eac72 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:21.126498 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e603a50-6f29-4d26-8678-420c0f5eac72 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:21.126776 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e603a50-6f29-4d26-8678-420c0f5eac72 None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:21.127531 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e603a50-6f29-4d26-8678-420c0f5eac72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:21.127875 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e603a50-6f29-4d26-8678-420c0f5eac72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:21.129296 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e603a50-6f29-4d26-8678-420c0f5eac72 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:21.149692 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 10/21] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:14:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:14:23.396953 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aff5a8-58ac-4046-b839-1f466a42249c None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:23.397811 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aff5a8-58ac-4046-b839-1f466a42249c None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:23.398237 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aff5a8-58ac-4046-b839-1f466a42249c None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:23.399801 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 12/22] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:14:23.405144 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e57af1-3fb5-4048-b86c-95c49999cb39 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:23.405878 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e57af1-3fb5-4048-b86c-95c49999cb39 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:23.406311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e57af1-3fb5-4048-b86c-95c49999cb39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:23.427836 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-59e57af1-3fb5-4048-b86c-95c49999cb39 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:23.429285 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-59e57af1-3fb5-4048-b86c-95c49999cb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:23.513289 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bdf846-7bcf-437a-8c0d-aa21dad23135 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:23.513802 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bdf846-7bcf-437a-8c0d-aa21dad23135 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:23.514228 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bdf846-7bcf-437a-8c0d-aa21dad23135 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:23.515760 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 13/23] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:14:23.521669 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c71bbb-c463-468d-8ace-b73162f252f9 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:23.522220 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c71bbb-c463-468d-8ace-b73162f252f9 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:23.522681 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c71bbb-c463-468d-8ace-b73162f252f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:23.525327 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 11/24] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:14:23.545693 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-95c71bbb-c463-468d-8ace-b73162f252f9 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:23.547167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-95c71bbb-c463-468d-8ace-b73162f252f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:23.560666 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e57af1-3fb5-4048-b86c-95c49999cb39 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:23.648817 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:23.676945 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 14/25] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:14:23.703954 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c71bbb-c463-468d-8ace-b73162f252f9 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:23.733649 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e57af1-3fb5-4048-b86c-95c49999cb39 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:23.797483 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86c9bc-b9c4-43ca-afd1-66dba7630507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:23.798612 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86c9bc-b9c4-43ca-afd1-66dba7630507 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:23.799056 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86c9bc-b9c4-43ca-afd1-66dba7630507 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:23.799678 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86c9bc-b9c4-43ca-afd1-66dba7630507 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:23.821653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f86c9bc-b9c4-43ca-afd1-66dba7630507 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:23.823148 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f86c9bc-b9c4-43ca-afd1-66dba7630507 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:23.843920 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:23.857201 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-6f86c9bc-b9c4-43ca-afd1-66dba7630507 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:14:23.857201 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:14:23.857201 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:14:23.857201 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:14:23.857201 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:14:23.857201 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:14:23.857201 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:14:23.857201 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:14:23.857201 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:14:23.857201 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:14:23.857201 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:14:23.857201 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:14:23.857201 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:14:23.857201 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:14:23.857201 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:14:23.862613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:14:23.862613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:14:23.862613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:14:23.862613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:14:23.862613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:14:23.862613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:14:23.862613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:14:23.862613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:14:23.862613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:14:23.862613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:14:23.862613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:14:23.862613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:14:23.862613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:14:23.862613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:14:23.862613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:14:23.862613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:14:23.862613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:14:23.863806 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:14:23.863806 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:14:23.863806 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:14:23.863806 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:14:23.863806 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 12/26] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:14:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:14:23.864303 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f093231b-2bd5-47a8-8a2a-768a903feb89 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:23.864443 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f093231b-2bd5-47a8-8a2a-768a903feb89 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:23.868355 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f093231b-2bd5-47a8-8a2a-768a903feb89 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:23.869851 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 13/27] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:14:23.871737 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f093231b-2bd5-47a8-8a2a-768a903feb89 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:23.890523 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c71bbb-c463-468d-8ace-b73162f252f9 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:23.930010 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7c774-b6ec-444a-8afb-fba192d8e418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:23.931058 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7c774-b6ec-444a-8afb-fba192d8e418 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:23.931499 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7c774-b6ec-444a-8afb-fba192d8e418 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:23.932121 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7c774-b6ec-444a-8afb-fba192d8e418 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:23.944975 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f7c774-b6ec-444a-8afb-fba192d8e418 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:14:23.954407 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-c5f7c774-b6ec-444a-8afb-fba192d8e418 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:14:23.954407 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:14:23.954407 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:14:23.954407 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:14:23.954407 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:14:23.954407 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:14:23.954407 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:14:23.954407 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:14:23.954407 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:14:23.954407 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:14:23.954407 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:14:23.954407 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:14:23.954407 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:14:23.954407 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:14:23.954407 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:14:23.954407 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:14:23.955297 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:14:23.955297 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:14:23.955297 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:14:23.955297 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:14:23.955297 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:14:23.955297 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:14:23.955297 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:14:23.955297 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:14:23.955297 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:14:23.955297 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:14:23.955297 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:14:23.955297 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:14:23.955297 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:14:23.955297 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:14:23.955297 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:14:23.955297 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:14:23.956710 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:14:23.956710 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:14:23.956710 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:14:23.956710 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:14:23.956710 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:14:23.956710 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:14:23.956710 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:14:23.956710 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:14:23.956710 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:14:23.956710 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:14:23.956710 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:14:23.956710 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:14:23.958750 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 15/28] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Jul 7 15:14:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:14:23.960687 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e69cbb-83ac-43f3-91cf-d22e684251fa None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:23.961613 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e69cbb-83ac-43f3-91cf-d22e684251fa None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:23.962027 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e69cbb-83ac-43f3-91cf-d22e684251fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:23.983241 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-62e69cbb-83ac-43f3-91cf-d22e684251fa None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:23.985474 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-62e69cbb-83ac-43f3-91cf-d22e684251fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:24.085244 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:24.090557 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f093231b-2bd5-47a8-8a2a-768a903feb89 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:24.115451 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 16/29] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:14:24.116623 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e69cbb-83ac-43f3-91cf-d22e684251fa None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:24.127717 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f81295-ed5c-486a-869a-f995a961361b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:24.129122 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f81295-ed5c-486a-869a-f995a961361b None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:24.129603 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f81295-ed5c-486a-869a-f995a961361b None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:24.130050 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f81295-ed5c-486a-869a-f995a961361b None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:24.130987 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f81295-ed5c-486a-869a-f995a961361b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:24.131453 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f81295-ed5c-486a-869a-f995a961361b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:24.133448 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f81295-ed5c-486a-869a-f995a961361b None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:24.146439 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 14/30] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:14:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:24.148048 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7c3cbe-77fc-4431-b3d6-6f9bb6f036f3 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:24.148532 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7c3cbe-77fc-4431-b3d6-6f9bb6f036f3 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:24.148972 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7c3cbe-77fc-4431-b3d6-6f9bb6f036f3 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:24.150398 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 15/31] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:14:24.158550 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5d5a3-0f64-4ce1-b506-e4b706843f48 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:24.158550 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5d5a3-0f64-4ce1-b506-e4b706843f48 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:24.158550 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef5d5a3-0f64-4ce1-b506-e4b706843f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:24.181132 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-bef5d5a3-0f64-4ce1-b506-e4b706843f48 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:24.182253 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-bef5d5a3-0f64-4ce1-b506-e4b706843f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:24.208094 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:24.247487 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e69cbb-83ac-43f3-91cf-d22e684251fa None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:24.275666 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 16/32] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:14:24.276533 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef5d5a3-0f64-4ce1-b506-e4b706843f48 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:24.297342 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8010e7-a249-4ad8-b1d8-08efb4844969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:24.298025 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8010e7-a249-4ad8-b1d8-08efb4844969 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:24.298308 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8010e7-a249-4ad8-b1d8-08efb4844969 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:24.298581 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8010e7-a249-4ad8-b1d8-08efb4844969 None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:24.299358 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8010e7-a249-4ad8-b1d8-08efb4844969 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:24.300911 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8010e7-a249-4ad8-b1d8-08efb4844969 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:24.302307 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8010e7-a249-4ad8-b1d8-08efb4844969 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:24.312905 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 17/33] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:14:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:24.313502 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8010e7-a249-4ad8-b1d8-08efb4844969 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:24.406116 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:24.421359 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef5d5a3-0f64-4ce1-b506-e4b706843f48 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:24.428412 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8010e7-a249-4ad8-b1d8-08efb4844969 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:24.457953 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34728d-2f57-4094-a0da-833bb9706408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:24.458705 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34728d-2f57-4094-a0da-833bb9706408 None admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:24.459089 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34728d-2f57-4094-a0da-833bb9706408 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:24.459488 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34728d-2f57-4094-a0da-833bb9706408 None admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:24.459943 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:24.460639 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c34728d-2f57-4094-a0da-833bb9706408 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:24.462266 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c34728d-2f57-4094-a0da-833bb9706408 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:24.468527 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d50a6-47e5-4778-be83-2e870a73f7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:24.469394 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d50a6-47e5-4778-be83-2e870a73f7d8 None admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:24.469820 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d50a6-47e5-4778-be83-2e870a73f7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:24.470228 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d50a6-47e5-4778-be83-2e870a73f7d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:24.471106 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8d50a6-47e5-4778-be83-2e870a73f7d8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:24.473559 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8d50a6-47e5-4778-be83-2e870a73f7d8 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:24.687247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 18/34] 10.222.0.25 () {60 vars in 1131 bytes} [Thu Jul 7 15:14:24 2022] POST /identity/v3/roles => generated 219 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:14:24.688593 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d50a6-47e5-4778-be83-2e870a73f7d8 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:24.757282 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 17/35] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Jul 7 15:14:24 2022] POST /identity/v3/projects => generated 301 bytes in 481 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:14:24.764519 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34728d-2f57-4094-a0da-833bb9706408 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:24.953025 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d50a6-47e5-4778-be83-2e870a73f7d8 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:24.997193 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cfc94a-af7e-4c60-b032-0cd4e3c3866b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:24.998031 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cfc94a-af7e-4c60-b032-0cd4e3c3866b None admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:24.998415 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cfc94a-af7e-4c60-b032-0cd4e3c3866b None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:24.998793 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cfc94a-af7e-4c60-b032-0cd4e3c3866b None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:24.999791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cfc94a-af7e-4c60-b032-0cd4e3c3866b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:25.001325 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cfc94a-af7e-4c60-b032-0cd4e3c3866b None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:25.015491 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34728d-2f57-4094-a0da-833bb9706408 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:25.057236 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3cdd01-e6f2-407d-ab00-7f8de0c33ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:25.058147 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3cdd01-e6f2-407d-ab00-7f8de0c33ffb None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:25.058428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3cdd01-e6f2-407d-ab00-7f8de0c33ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:25.058703 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3cdd01-e6f2-407d-ab00-7f8de0c33ffb None admin] PATH_INFO: `/v3/users/30014035e7d641d5aecffdbe884d3d52` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:25.060731 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3cdd01-e6f2-407d-ab00-7f8de0c33ffb None admin] RBAC: Authorizing `identity:get_user(user_id=30014035e7d641d5aecffdbe884d3d52)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:25.062400 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3cdd01-e6f2-407d-ab00-7f8de0c33ffb None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:25.064363 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 18/36] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:14:24 2022] GET /identity/v3/users/30014035e7d641d5aecffdbe884d3d52 => generated 247 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:25.066153 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d517608-6a6b-468d-b681-354032cb9822 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:25.066441 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d517608-6a6b-468d-b681-354032cb9822 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:25.066715 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d517608-6a6b-468d-b681-354032cb9822 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:25.068286 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 19/37] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:14:25.072315 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde195a-5c38-4365-b3df-49d5280b02b8 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:25.072576 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde195a-5c38-4365-b3df-49d5280b02b8 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:25.072852 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde195a-5c38-4365-b3df-49d5280b02b8 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:25.073334 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 19/38] 10.222.0.25 () {60 vars in 1131 bytes} [Thu Jul 7 15:14:24 2022] POST /identity/v3/roles => generated 225 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:14:25.074639 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 20/39] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:14:25.076049 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed58044-9230-4296-9cdd-c9aabe6259a9 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:25.076341 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed58044-9230-4296-9cdd-c9aabe6259a9 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:25.076621 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed58044-9230-4296-9cdd-c9aabe6259a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:25.079538 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde195a-5c38-4365-b3df-49d5280b02b8 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:25.095860 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:25.105508 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-2ed58044-9230-4296-9cdd-c9aabe6259a9 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:25.107168 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-2ed58044-9230-4296-9cdd-c9aabe6259a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:25.168409 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:25.203590 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 20/40] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 07 15:14:25.203590 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871af1a9-4360-439f-ae5e-7c0904d6d2ab None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:25.203590 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871af1a9-4360-439f-ae5e-7c0904d6d2ab None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:25.203590 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871af1a9-4360-439f-ae5e-7c0904d6d2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:25.217040 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-871af1a9-4360-439f-ae5e-7c0904d6d2ab None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:25.218758 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-871af1a9-4360-439f-ae5e-7c0904d6d2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:25.229456 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde195a-5c38-4365-b3df-49d5280b02b8 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:25.271325 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4280b0-ddb4-49a0-94e7-fd4a015fb44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:25.272090 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4280b0-ddb4-49a0-94e7-fd4a015fb44e None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:25.272373 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4280b0-ddb4-49a0-94e7-fd4a015fb44e None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:25.272762 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4280b0-ddb4-49a0-94e7-fd4a015fb44e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:25.274092 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4280b0-ddb4-49a0-94e7-fd4a015fb44e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:25.275749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4280b0-ddb4-49a0-94e7-fd4a015fb44e None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:25.277376 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 21/41] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:14:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:25.278195 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4280b0-ddb4-49a0-94e7-fd4a015fb44e None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:25.298929 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 21/42] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:14:25.300340 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871af1a9-4360-439f-ae5e-7c0904d6d2ab None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:25.357223 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:25.380313 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4280b0-ddb4-49a0-94e7-fd4a015fb44e None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:25.406007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871af1a9-4360-439f-ae5e-7c0904d6d2ab None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:25.416633 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f53d2fb-d5bb-4157-bd60-ac6a161241df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:25.417330 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f53d2fb-d5bb-4157-bd60-ac6a161241df None admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:25.417622 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f53d2fb-d5bb-4157-bd60-ac6a161241df None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:25.417904 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f53d2fb-d5bb-4157-bd60-ac6a161241df None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:25.418688 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f53d2fb-d5bb-4157-bd60-ac6a161241df None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:25.420154 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f53d2fb-d5bb-4157-bd60-ac6a161241df None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:25.509315 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 22/43] 10.222.0.25 () {60 vars in 1131 bytes} [Thu Jul 7 15:14:25 2022] POST /identity/v3/roles => generated 223 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:14:25.510172 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f53d2fb-d5bb-4157-bd60-ac6a161241df None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:25.519558 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:25.555884 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0fd2b6-0f0b-40db-9cc1-18d200686dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:25.556694 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0fd2b6-0f0b-40db-9cc1-18d200686dd6 None admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:25.556984 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0fd2b6-0f0b-40db-9cc1-18d200686dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:25.557270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0fd2b6-0f0b-40db-9cc1-18d200686dd6 None admin] PATH_INFO: `/v3/domains/default/users/30014035e7d641d5aecffdbe884d3d52/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:25.559447 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0fd2b6-0f0b-40db-9cc1-18d200686dd6 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=30014035e7d641d5aecffdbe884d3d52, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:25.561029 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0fd2b6-0f0b-40db-9cc1-18d200686dd6 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:25.635220 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f53d2fb-d5bb-4157-bd60-ac6a161241df None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:25.649122 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 22/44] 10.222.0.25 () {58 vars in 1361 bytes} [Thu Jul 7 15:14:25 2022] PUT /identity/v3/domains/default/users/30014035e7d641d5aecffdbe884d3d52/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:14:25.676266 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae58bb3-ff13-4890-9763-ba024c5c3d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:25.677085 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae58bb3-ff13-4890-9763-ba024c5c3d1f None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:25.677379 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae58bb3-ff13-4890-9763-ba024c5c3d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:25.677655 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae58bb3-ff13-4890-9763-ba024c5c3d1f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:25.678922 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae58bb3-ff13-4890-9763-ba024c5c3d1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:25.680524 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae58bb3-ff13-4890-9763-ba024c5c3d1f None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:25.682250 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 23/45] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:14:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:25.688044 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0fd2b6-0f0b-40db-9cc1-18d200686dd6 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:25.818651 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0fd2b6-0f0b-40db-9cc1-18d200686dd6 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:25.853108 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee0da5-3007-4bc9-832b-f94bc280a2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:25.853770 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee0da5-3007-4bc9-832b-f94bc280a2ad None admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:25.854056 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee0da5-3007-4bc9-832b-f94bc280a2ad None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:25.854336 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee0da5-3007-4bc9-832b-f94bc280a2ad None admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:25.855061 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee0da5-3007-4bc9-832b-f94bc280a2ad None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:25.856527 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee0da5-3007-4bc9-832b-f94bc280a2ad None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:25.998343 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 23/46] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Jul 7 15:14:25 2022] POST /identity/v3/projects => generated 312 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:14:27.685616 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baedb68-6b7c-49b2-af0f-df2f7dce823a None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:27.686129 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baedb68-6b7c-49b2-af0f-df2f7dce823a None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:27.686212 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baedb68-6b7c-49b2-af0f-df2f7dce823a None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:27.687530 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 24/47] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:14:27.692865 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d79d1-031f-41ce-b58b-b728235d0ef2 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:27.693286 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d79d1-031f-41ce-b58b-b728235d0ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:27.693432 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d79d1-031f-41ce-b58b-b728235d0ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:27.710559 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-726d79d1-031f-41ce-b58b-b728235d0ef2 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:27.711793 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-726d79d1-031f-41ce-b58b-b728235d0ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:27.727078 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:27.779662 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 24/48] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:14:27.803077 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baedb68-6b7c-49b2-af0f-df2f7dce823a None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:27.807004 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:28.012518 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baedb68-6b7c-49b2-af0f-df2f7dce823a None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:28.046197 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b901988d-cf31-42eb-a4a8-a87c4d0a0d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:28.046986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901988d-cf31-42eb-a4a8-a87c4d0a0d22 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:28.047275 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901988d-cf31-42eb-a4a8-a87c4d0a0d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:28.047553 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901988d-cf31-42eb-a4a8-a87c4d0a0d22 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:28.058946 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b901988d-cf31-42eb-a4a8-a87c4d0a0d22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:14:28.067797 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-b901988d-cf31-42eb-a4a8-a87c4d0a0d22 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:14:28.067797 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:14:28.067797 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:14:28.067797 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:14:28.067797 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:14:28.067797 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:14:28.067797 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:14:28.067797 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:14:28.067797 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:14:28.067797 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:14:28.067797 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:14:28.067797 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:14:28.067797 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:14:28.067797 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:14:28.067797 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:14:28.067797 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:14:28.068764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:14:28.068764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:14:28.068764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:14:28.068764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:14:28.068764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:14:28.068764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:14:28.068764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:14:28.068764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:14:28.068764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:14:28.068764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:14:28.068764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:14:28.068764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:14:28.068764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:14:28.068764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:14:28.068764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:14:28.068764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:14:28.069714 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:14:28.069714 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:14:28.069714 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:14:28.069714 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:14:28.069714 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:14:28.069714 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:14:28.069714 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:14:28.069714 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:14:28.069714 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:14:28.069714 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:14:28.069714 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:14:28.069714 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:14:28.069714 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 25/49] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Jul 7 15:14:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:14:28.076431 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d79d1-031f-41ce-b58b-b728235d0ef2 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:28.100626 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ec50b-489a-43d4-8c1b-4545312448ef None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:28.101033 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ec50b-489a-43d4-8c1b-4545312448ef None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:28.101317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ec50b-489a-43d4-8c1b-4545312448ef None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:28.102683 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 26/50] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:14:28.107517 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950f673-12b0-49e7-9eb0-1db5943ec652 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:28.107841 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950f673-12b0-49e7-9eb0-1db5943ec652 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:28.108129 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950f673-12b0-49e7-9eb0-1db5943ec652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:28.124856 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-c950f673-12b0-49e7-9eb0-1db5943ec652 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:28.126399 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-c950f673-12b0-49e7-9eb0-1db5943ec652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:28.176360 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d79d1-031f-41ce-b58b-b728235d0ef2 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:28.185131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:28.194002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 27/51] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 15:14:28.210895 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53301fe6-edb3-443f-95f7-e2dfb27e15ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:28.211579 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53301fe6-edb3-443f-95f7-e2dfb27e15ef None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:28.211878 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53301fe6-edb3-443f-95f7-e2dfb27e15ef None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:28.212157 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53301fe6-edb3-443f-95f7-e2dfb27e15ef None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:28.212910 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53301fe6-edb3-443f-95f7-e2dfb27e15ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:28.213217 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53301fe6-edb3-443f-95f7-e2dfb27e15ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:28.214688 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53301fe6-edb3-443f-95f7-e2dfb27e15ef None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:28.217311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c950f673-12b0-49e7-9eb0-1db5943ec652 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:28.223211 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 25/52] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:14:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:28.228347 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53301fe6-edb3-443f-95f7-e2dfb27e15ef None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:28.240148 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:28.317486 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c950f673-12b0-49e7-9eb0-1db5943ec652 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:28.337114 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53301fe6-edb3-443f-95f7-e2dfb27e15ef None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:28.351273 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72d1867-e761-44dc-8a5d-faded54c1779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:28.352088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d1867-e761-44dc-8a5d-faded54c1779 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:28.352373 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d1867-e761-44dc-8a5d-faded54c1779 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:28.352656 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d1867-e761-44dc-8a5d-faded54c1779 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:28.353838 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72d1867-e761-44dc-8a5d-faded54c1779 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:28.355287 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72d1867-e761-44dc-8a5d-faded54c1779 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:28.356976 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 28/53] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:14:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:14:28.362392 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72d1867-e761-44dc-8a5d-faded54c1779 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:28.371878 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951efa1-c9ac-4353-a8b1-51125b0ea712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:28.372749 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951efa1-c9ac-4353-a8b1-51125b0ea712 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:28.373027 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951efa1-c9ac-4353-a8b1-51125b0ea712 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:28.373297 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951efa1-c9ac-4353-a8b1-51125b0ea712 None admin] PATH_INFO: `/v3/users/30014035e7d641d5aecffdbe884d3d52` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:28.374493 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5951efa1-c9ac-4353-a8b1-51125b0ea712 None admin] RBAC: Authorizing `identity:get_user(user_id=30014035e7d641d5aecffdbe884d3d52)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:28.375954 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5951efa1-c9ac-4353-a8b1-51125b0ea712 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:28.378095 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 26/54] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:14:28 2022] GET /identity/v3/users/30014035e7d641d5aecffdbe884d3d52 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:14:28.383272 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951efa1-c9ac-4353-a8b1-51125b0ea712 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:28.423105 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:28.459644 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:28.462522 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72d1867-e761-44dc-8a5d-faded54c1779 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:28.480538 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951efa1-c9ac-4353-a8b1-51125b0ea712 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:28.496698 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3caff4-98d1-4687-884b-979575e79a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:28.497350 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3caff4-98d1-4687-884b-979575e79a56 None admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:28.497633 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3caff4-98d1-4687-884b-979575e79a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:28.497909 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3caff4-98d1-4687-884b-979575e79a56 None admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:28.498617 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3caff4-98d1-4687-884b-979575e79a56 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:28.500087 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3caff4-98d1-4687-884b-979575e79a56 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:28.517321 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1334ea-8f4b-402f-a1fa-fd3595e1eee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:28.518114 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1334ea-8f4b-402f-a1fa-fd3595e1eee8 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:28.518419 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1334ea-8f4b-402f-a1fa-fd3595e1eee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:28.518697 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1334ea-8f4b-402f-a1fa-fd3595e1eee8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:28.519909 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1334ea-8f4b-402f-a1fa-fd3595e1eee8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:28.521373 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1334ea-8f4b-402f-a1fa-fd3595e1eee8 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:28.522990 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 27/55] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:14:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:14:28.528796 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1334ea-8f4b-402f-a1fa-fd3595e1eee8 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:28.589414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 29/56] 10.222.0.25 () {60 vars in 1140 bytes} [Thu Jul 7 15:14:28 2022] POST /identity/v3/projects => generated 298 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:14:28.630197 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1334ea-8f4b-402f-a1fa-fd3595e1eee8 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:28.665929 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d94abd-b684-42e5-b14e-ac2fd2dd226f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:28.666611 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d94abd-b684-42e5-b14e-ac2fd2dd226f None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:28.666900 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d94abd-b684-42e5-b14e-ac2fd2dd226f None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:28.667182 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d94abd-b684-42e5-b14e-ac2fd2dd226f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:28.668079 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d94abd-b684-42e5-b14e-ac2fd2dd226f None admin] RBAC: Adding query filter params (role.id=0e7793fbf3064037984da651879e3a94, scope.domain.id=default, user.id=30014035e7d641d5aecffdbe884d3d52) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:28.668400 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d94abd-b684-42e5-b14e-ac2fd2dd226f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:28.669894 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d94abd-b684-42e5-b14e-ac2fd2dd226f None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:28.677027 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 28/57] 10.222.0.25 () {56 vars in 1322 bytes} [Thu Jul 7 15:14:28 2022] GET /identity/v3/role_assignments?user.id=30014035e7d641d5aecffdbe884d3d52&scope.domain.id=default&role.id=0e7793fbf3064037984da651879e3a94 => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:14:30.616575 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f977474-6f15-4946-91d2-a0a091a6b7d3 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:30.617092 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f977474-6f15-4946-91d2-a0a091a6b7d3 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:30.617198 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f977474-6f15-4946-91d2-a0a091a6b7d3 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:30.618519 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 30/58] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:14:30.623458 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4f3d0-cdd2-4e60-b6c7-319420ab76af None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:30.623895 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4f3d0-cdd2-4e60-b6c7-319420ab76af None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:30.624045 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4f3d0-cdd2-4e60-b6c7-319420ab76af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:30.641023 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-68d4f3d0-cdd2-4e60-b6c7-319420ab76af None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:30.642222 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-68d4f3d0-cdd2-4e60-b6c7-319420ab76af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:30.645553 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:30.706659 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 29/59] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:14:30.730755 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f977474-6f15-4946-91d2-a0a091a6b7d3 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:30.835406 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f977474-6f15-4946-91d2-a0a091a6b7d3 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:30.849493 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:30.869140 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7eaded-e7ca-4048-b239-8f5096af490f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:30.869872 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7eaded-e7ca-4048-b239-8f5096af490f None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:30.870185 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7eaded-e7ca-4048-b239-8f5096af490f None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:30.870467 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7eaded-e7ca-4048-b239-8f5096af490f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:30.871679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7eaded-e7ca-4048-b239-8f5096af490f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:30.873133 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7eaded-e7ca-4048-b239-8f5096af490f None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:30.874750 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 31/60] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:14:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:30.880796 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d4f3d0-cdd2-4e60-b6c7-319420ab76af None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:30.977359 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d4f3d0-cdd2-4e60-b6c7-319420ab76af None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:31.011045 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd10ca-4e27-45d0-84ca-0a14fe417eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:31.011734 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd10ca-4e27-45d0-84ca-0a14fe417eb9 None admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:31.012020 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd10ca-4e27-45d0-84ca-0a14fe417eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:31.012295 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd10ca-4e27-45d0-84ca-0a14fe417eb9 None admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:31.013008 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd10ca-4e27-45d0-84ca-0a14fe417eb9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:31.014538 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd10ca-4e27-45d0-84ca-0a14fe417eb9 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:31.029154 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:31.123321 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 30/61] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Jul 7 15:14:30 2022] POST /identity/v3/users => generated 275 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:14:34.308241 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f3425-48f5-4f3f-ab4e-b754a690bec6 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:34.308241 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f3425-48f5-4f3f-ab4e-b754a690bec6 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:34.308241 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f3425-48f5-4f3f-ab4e-b754a690bec6 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:34.309331 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 32/62] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:14:34.313219 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c33674-3b11-4e79-b27c-38871782c150 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:34.314003 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c33674-3b11-4e79-b27c-38871782c150 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:34.314574 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c33674-3b11-4e79-b27c-38871782c150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:34.335956 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f486d-68f9-4577-9beb-bbd484be2d2d None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:34.336553 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f486d-68f9-4577-9beb-bbd484be2d2d None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:34.337000 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f486d-68f9-4577-9beb-bbd484be2d2d None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:34.338778 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 33/63] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:14:34.339363 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-20c33674-3b11-4e79-b27c-38871782c150 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:34.340919 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-20c33674-3b11-4e79-b27c-38871782c150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:34.352002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5490671-1343-4ea1-9575-cafd52c2a28a None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:34.352904 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5490671-1343-4ea1-9575-cafd52c2a28a None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:34.353624 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5490671-1343-4ea1-9575-cafd52c2a28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:34.383681 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-b5490671-1343-4ea1-9575-cafd52c2a28a None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:34.388411 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-b5490671-1343-4ea1-9575-cafd52c2a28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:34.490774 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 31/64] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:14:34.507868 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e94cbe-824c-4e5b-8c90-378ed16e6c3a None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:34.508450 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e94cbe-824c-4e5b-8c90-378ed16e6c3a None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:34.508948 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e94cbe-824c-4e5b-8c90-378ed16e6c3a None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:34.510513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 32/65] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:14:34.511226 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 34/66] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 15:14:34.515002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5490671-1343-4ea1-9575-cafd52c2a28a None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:34.524338 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7797316-13fa-4214-80da-0dba06211e7b None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:34.524897 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7797316-13fa-4214-80da-0dba06211e7b None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:34.525368 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7797316-13fa-4214-80da-0dba06211e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:34.553810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-e7797316-13fa-4214-80da-0dba06211e7b None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:34.555324 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-e7797316-13fa-4214-80da-0dba06211e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:34.625099 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 33/67] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 15:14:34.626479 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7797316-13fa-4214-80da-0dba06211e7b None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:34.650761 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5490671-1343-4ea1-9575-cafd52c2a28a None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:34.670282 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:34.692280 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:34.694110 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b538868-f0ec-4c38-ad3e-d07efdb5a991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:34.694904 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b538868-f0ec-4c38-ad3e-d07efdb5a991 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:34.695189 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b538868-f0ec-4c38-ad3e-d07efdb5a991 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:34.695465 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b538868-f0ec-4c38-ad3e-d07efdb5a991 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:34.706432 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b538868-f0ec-4c38-ad3e-d07efdb5a991 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:14:34.714365 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-1b538868-f0ec-4c38-ad3e-d07efdb5a991 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:14:34.714365 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:14:34.714365 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:14:34.714365 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:14:34.714365 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:14:34.714365 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:14:34.714365 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:14:34.714365 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:14:34.714365 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:14:34.714365 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:14:34.714365 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:14:34.714365 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:14:34.714365 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:14:34.714365 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:14:34.714365 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:14:34.714365 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:14:34.715422 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:14:34.715422 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:14:34.715422 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:14:34.715422 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:14:34.715422 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:14:34.715422 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:14:34.715422 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:14:34.715422 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:14:34.715422 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:14:34.715422 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:14:34.715422 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:14:34.715422 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:14:34.715422 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:14:34.715422 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:14:34.715422 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:14:34.715422 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:14:34.723122 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:14:34.723122 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:14:34.723122 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:14:34.723122 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:14:34.723122 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:14:34.723122 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:14:34.723122 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:14:34.723122 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:14:34.723122 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:14:34.723122 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:14:34.723122 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:14:34.723122 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:14:34.723122 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 35/68] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Jul 7 15:14:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:14:34.723122 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b538868-f0ec-4c38-ad3e-d07efdb5a991 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:34.742574 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7797316-13fa-4214-80da-0dba06211e7b None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:34.778676 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcda02ac-e2b5-4bdc-b727-7d1414017b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:34.779770 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda02ac-e2b5-4bdc-b727-7d1414017b57 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:34.780227 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda02ac-e2b5-4bdc-b727-7d1414017b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:34.780676 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda02ac-e2b5-4bdc-b727-7d1414017b57 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:34.792814 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcda02ac-e2b5-4bdc-b727-7d1414017b57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:14:34.800612 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-dcda02ac-e2b5-4bdc-b727-7d1414017b57 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 07 15:14:34.800612 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:14:34.800612 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:14:34.800612 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:14:34.800612 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:14:34.800612 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:14:34.800612 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:14:34.800612 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:14:34.800612 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:14:34.800612 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:14:34.800612 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:14:34.800612 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:14:34.800612 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:14:34.800612 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:14:34.800612 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:14:34.800612 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:14:34.801499 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:14:34.801499 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:14:34.801499 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:14:34.801499 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:14:34.801499 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:14:34.801499 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:14:34.801499 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:14:34.801499 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:14:34.801499 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:14:34.801499 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:14:34.801499 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:14:34.801499 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:14:34.801499 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:14:34.801499 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:14:34.801499 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:14:34.801499 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:14:34.807246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:14:34.807246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:14:34.807246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:14:34.807246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:14:34.807246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:14:34.807246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:14:34.807246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:14:34.807246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:14:34.807246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:14:34.807246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:14:34.807246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 07 15:14:34.807246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:14:34.807246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 34/69] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:14:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:14:34.807246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62493b-5c06-455f-846a-29d473b6e63f None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:34.807956 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62493b-5c06-455f-846a-29d473b6e63f None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:34.807956 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62493b-5c06-455f-846a-29d473b6e63f None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:34.809189 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 35/70] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:14:34.809992 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62493b-5c06-455f-846a-29d473b6e63f None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:34.831380 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b538868-f0ec-4c38-ad3e-d07efdb5a991 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:34.865195 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac19e83-cc68-4d00-b50f-4e09eaa4a533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:34.865997 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac19e83-cc68-4d00-b50f-4e09eaa4a533 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:34.866283 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac19e83-cc68-4d00-b50f-4e09eaa4a533 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:34.866564 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac19e83-cc68-4d00-b50f-4e09eaa4a533 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:34.876560 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac19e83-cc68-4d00-b50f-4e09eaa4a533 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:14:34.885049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-5ac19e83-cc68-4d00-b50f-4e09eaa4a533 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 15:14:34.885049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:14:34.885049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:14:34.885049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:14:34.885049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:14:34.885049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:14:34.885049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:14:34.885049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:14:34.885049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:14:34.885049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:14:34.885049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:14:34.885049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:14:34.885049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:14:34.885049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:14:34.885049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:14:34.885049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:14:34.885875 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:14:34.885875 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:14:34.885875 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:14:34.885875 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:14:34.885875 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:14:34.885875 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:14:34.885875 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:14:34.885875 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:14:34.885875 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:14:34.885875 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:14:34.885875 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:14:34.885875 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:14:34.885875 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:14:34.885875 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:14:34.885875 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:14:34.885875 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:14:34.886654 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:14:34.886654 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:14:34.886654 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:14:34.886654 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:14:34.886654 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:14:34.886654 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:14:34.886654 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:14:34.886654 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:14:34.886654 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:14:34.886654 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:14:34.886654 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 15:14:34.886654 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:14:34.886654 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 36/71] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Jul 7 15:14:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:14:34.888148 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac19e83-cc68-4d00-b50f-4e09eaa4a533 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:34.911475 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62493b-5c06-455f-846a-29d473b6e63f None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:34.918519 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:34.926922 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:34.944128 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53615303-e10c-460d-baf5-215905880fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:34.944876 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53615303-e10c-460d-baf5-215905880fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:34.945284 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53615303-e10c-460d-baf5-215905880fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:34.945570 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53615303-e10c-460d-baf5-215905880fc6 None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:34.946320 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53615303-e10c-460d-baf5-215905880fc6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:34.946653 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53615303-e10c-460d-baf5-215905880fc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:34.948096 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53615303-e10c-460d-baf5-215905880fc6 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:34.957312 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 36/72] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:14:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:34.958615 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fdc773-4512-45a5-9bf7-7802d7bc5771 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:34.958900 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fdc773-4512-45a5-9bf7-7802d7bc5771 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:34.959175 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fdc773-4512-45a5-9bf7-7802d7bc5771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:34.976645 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-71fdc773-4512-45a5-9bf7-7802d7bc5771 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:34.977852 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-71fdc773-4512-45a5-9bf7-7802d7bc5771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:34.991427 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac19e83-cc68-4d00-b50f-4e09eaa4a533 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:35.025476 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffa4995-02b5-4fae-80d8-153d28410f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:35.026132 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa4995-02b5-4fae-80d8-153d28410f3f None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:35.026417 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa4995-02b5-4fae-80d8-153d28410f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:35.026695 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa4995-02b5-4fae-80d8-153d28410f3f None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:35.027440 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffa4995-02b5-4fae-80d8-153d28410f3f None admin] RBAC: Adding query filter params (name=member) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:35.027789 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffa4995-02b5-4fae-80d8-153d28410f3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:35.029159 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffa4995-02b5-4fae-80d8-153d28410f3f None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:35.040253 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 37/73] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Jul 7 15:14:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:35.041525 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377f8c5-d064-4741-a473-5b5df6521dbe None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:35.041816 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377f8c5-d064-4741-a473-5b5df6521dbe None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:35.042095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377f8c5-d064-4741-a473-5b5df6521dbe None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:35.043315 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 38/74] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:14:35.044023 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b377f8c5-d064-4741-a473-5b5df6521dbe None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:35.046003 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 37/75] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:14:35.047052 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fdc773-4512-45a5-9bf7-7802d7bc5771 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:35.139347 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:35.145978 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fdc773-4512-45a5-9bf7-7802d7bc5771 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:35.148572 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b377f8c5-d064-4741-a473-5b5df6521dbe None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:35.159692 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:35.184962 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777510ef-9c48-471b-96e3-5039511fc3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:35.185632 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777510ef-9c48-471b-96e3-5039511fc3b4 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:35.185921 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777510ef-9c48-471b-96e3-5039511fc3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:35.186012 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c77d51-f0d2-4183-acc0-895aea3c6a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:35.186204 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777510ef-9c48-471b-96e3-5039511fc3b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:35.186848 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c77d51-f0d2-4183-acc0-895aea3c6a6e None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:35.186966 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777510ef-9c48-471b-96e3-5039511fc3b4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:35.187131 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c77d51-f0d2-4183-acc0-895aea3c6a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:35.187283 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777510ef-9c48-471b-96e3-5039511fc3b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:35.187405 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c77d51-f0d2-4183-acc0-895aea3c6a6e None admin] PATH_INFO: `/v3/users/30014035e7d641d5aecffdbe884d3d52` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:35.188622 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c77d51-f0d2-4183-acc0-895aea3c6a6e None admin] RBAC: Authorizing `identity:get_user(user_id=30014035e7d641d5aecffdbe884d3d52)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:35.188742 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777510ef-9c48-471b-96e3-5039511fc3b4 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:35.190722 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c77d51-f0d2-4183-acc0-895aea3c6a6e None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:35.192424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 38/76] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:14:35 2022] GET /identity/v3/users/30014035e7d641d5aecffdbe884d3d52 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:35.193171 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c77d51-f0d2-4183-acc0-895aea3c6a6e None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:35.200284 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 39/77] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:14:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:35.201546 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5cf44-9a0e-428f-a1ae-b49f2e7daa67 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:35.201843 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5cf44-9a0e-428f-a1ae-b49f2e7daa67 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:35.202127 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5cf44-9a0e-428f-a1ae-b49f2e7daa67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:35.218778 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-e7b5cf44-9a0e-428f-a1ae-b49f2e7daa67 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:35.220033 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-e7b5cf44-9a0e-428f-a1ae-b49f2e7daa67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:35.289974 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 40/78] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:14:35.290824 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b5cf44-9a0e-428f-a1ae-b49f2e7daa67 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:35.297701 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c77d51-f0d2-4183-acc0-895aea3c6a6e None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:35.332117 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3756e86f-1ee1-42e4-be7e-d8ea305c2c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:35.332993 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756e86f-1ee1-42e4-be7e-d8ea305c2c1c None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:35.333275 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756e86f-1ee1-42e4-be7e-d8ea305c2c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:35.333552 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756e86f-1ee1-42e4-be7e-d8ea305c2c1c None admin] PATH_INFO: `/v3/users/c4236ad3d5d949cf9d36da8484c8bf39` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:35.346417 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3756e86f-1ee1-42e4-be7e-d8ea305c2c1c None admin] RBAC: Authorizing `identity:get_user(user_id=c4236ad3d5d949cf9d36da8484c8bf39)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:35.348190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3756e86f-1ee1-42e4-be7e-d8ea305c2c1c None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:35.349829 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 39/79] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:14:35 2022] GET /identity/v3/users/c4236ad3d5d949cf9d36da8484c8bf39 => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:35.350579 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3756e86f-1ee1-42e4-be7e-d8ea305c2c1c None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:35.375160 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:35.387907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:35.390468 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b5cf44-9a0e-428f-a1ae-b49f2e7daa67 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:35.423133 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcbf7ae-2f18-4350-a2a2-6880c201471a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:35.423944 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcbf7ae-2f18-4350-a2a2-6880c201471a None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:35.424770 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcbf7ae-2f18-4350-a2a2-6880c201471a None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:35.425050 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcbf7ae-2f18-4350-a2a2-6880c201471a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:35.435185 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcbf7ae-2f18-4350-a2a2-6880c201471a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:14:35.443416 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-cdcbf7ae-2f18-4350-a2a2-6880c201471a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 07 15:14:35.443416 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:14:35.443416 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:14:35.443416 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:14:35.443416 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:14:35.443416 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:14:35.443416 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:14:35.443416 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:14:35.443416 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:14:35.443416 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:14:35.443416 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:14:35.443416 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:14:35.443416 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:14:35.443416 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:14:35.443416 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:14:35.443416 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:14:35.444302 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:14:35.444302 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:14:35.444302 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:14:35.444302 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:14:35.444302 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:14:35.444302 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:14:35.444302 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:14:35.444302 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:14:35.444302 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:14:35.444302 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:14:35.444302 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:14:35.444302 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:14:35.444302 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:14:35.444302 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:14:35.444302 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:14:35.444302 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:14:35.445113 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:14:35.445113 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:14:35.445113 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:14:35.445113 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:14:35.445113 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:14:35.445113 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:14:35.445113 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:14:35.445113 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:14:35.445113 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:14:35.445113 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:14:35.445113 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 07 15:14:35.445113 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:14:35.445113 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 41/80] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:14:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:14:35.445753 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcbf7ae-2f18-4350-a2a2-6880c201471a None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:35.456461 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3756e86f-1ee1-42e4-be7e-d8ea305c2c1c None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:35.490719 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d69000-a17d-415d-9724-1279cb0c609e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:35.491381 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d69000-a17d-415d-9724-1279cb0c609e None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:35.491691 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d69000-a17d-415d-9724-1279cb0c609e None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:35.491972 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d69000-a17d-415d-9724-1279cb0c609e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:35.562264 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d69000-a17d-415d-9724-1279cb0c609e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:35.563768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d69000-a17d-415d-9724-1279cb0c609e None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:35.566078 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcbf7ae-2f18-4350-a2a2-6880c201471a None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:35.596516 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:35.603886 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a468ac0-7464-4c69-9de7-0d6aa6c7e889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:35.606275 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a468ac0-7464-4c69-9de7-0d6aa6c7e889 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:35.606275 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a468ac0-7464-4c69-9de7-0d6aa6c7e889 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:35.606275 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a468ac0-7464-4c69-9de7-0d6aa6c7e889 None admin] PATH_INFO: `/v3/users/c4236ad3d5d949cf9d36da8484c8bf39` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:35.606743 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a468ac0-7464-4c69-9de7-0d6aa6c7e889 None admin] RBAC: Authorizing `identity:get_user(user_id=c4236ad3d5d949cf9d36da8484c8bf39)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:35.608217 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a468ac0-7464-4c69-9de7-0d6aa6c7e889 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:35.609836 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 42/81] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:14:35 2022] GET /identity/v3/users/c4236ad3d5d949cf9d36da8484c8bf39 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:35.610587 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a468ac0-7464-4c69-9de7-0d6aa6c7e889 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:35.630893 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:35.661253 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 40/82] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:14:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 15:14:35.664440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d69000-a17d-415d-9724-1279cb0c609e None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:35.715145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a468ac0-7464-4c69-9de7-0d6aa6c7e889 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:35.750686 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ed3745-ca4a-4271-8d5d-a3b91852cde2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:35.751468 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed3745-ca4a-4271-8d5d-a3b91852cde2 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:35.751784 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed3745-ca4a-4271-8d5d-a3b91852cde2 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:35.752063 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed3745-ca4a-4271-8d5d-a3b91852cde2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:35.753284 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ed3745-ca4a-4271-8d5d-a3b91852cde2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:35.754714 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ed3745-ca4a-4271-8d5d-a3b91852cde2 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:35.756349 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 43/83] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:14:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:35.757104 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ed3745-ca4a-4271-8d5d-a3b91852cde2 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:35.764551 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d69000-a17d-415d-9724-1279cb0c609e None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:35.798343 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012047f6-f104-49e5-86cd-0eca29501e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:35.798987 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012047f6-f104-49e5-86cd-0eca29501e87 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:35.799851 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012047f6-f104-49e5-86cd-0eca29501e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:35.800132 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012047f6-f104-49e5-86cd-0eca29501e87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:35.802735 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:35.839955 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:35.858298 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ed3745-ca4a-4271-8d5d-a3b91852cde2 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:35.875933 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012047f6-f104-49e5-86cd-0eca29501e87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:35.877544 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012047f6-f104-49e5-86cd-0eca29501e87 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:35.892555 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aef33c-83a9-43c6-8f0f-5fc21ed7ae1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:35.893223 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aef33c-83a9-43c6-8f0f-5fc21ed7ae1d None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:35.893511 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aef33c-83a9-43c6-8f0f-5fc21ed7ae1d None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:35.893799 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aef33c-83a9-43c6-8f0f-5fc21ed7ae1d None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:35.894744 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aef33c-83a9-43c6-8f0f-5fc21ed7ae1d None admin] RBAC: Adding query filter params (name=member) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:35.894855 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aef33c-83a9-43c6-8f0f-5fc21ed7ae1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:35.896301 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aef33c-83a9-43c6-8f0f-5fc21ed7ae1d None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:35.905520 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 44/84] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Jul 7 15:14:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:35.906375 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aef33c-83a9-43c6-8f0f-5fc21ed7ae1d None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:35.981356 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 41/85] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:14:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 15:14:35.982201 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012047f6-f104-49e5-86cd-0eca29501e87 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:36.014529 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:36.017326 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aef33c-83a9-43c6-8f0f-5fc21ed7ae1d None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:36.051288 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3c6a31-8781-4e79-9478-3a8fef78e3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:36.051969 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3c6a31-8781-4e79-9478-3a8fef78e3f8 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:36.052258 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3c6a31-8781-4e79-9478-3a8fef78e3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:36.052539 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3c6a31-8781-4e79-9478-3a8fef78e3f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:36.066927 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:36.082413 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012047f6-f104-49e5-86cd-0eca29501e87 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:36.119071 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4472f215-5368-4a3a-9914-2457d056e733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:36.120170 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472f215-5368-4a3a-9914-2457d056e733 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:36.120443 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472f215-5368-4a3a-9914-2457d056e733 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:36.120717 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472f215-5368-4a3a-9914-2457d056e733 None admin] PATH_INFO: `/v3/projects/db427256e2ec4ec084f762a615d0f187` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:36.121946 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4472f215-5368-4a3a-9914-2457d056e733 None admin] RBAC: Authorizing `identity:get_project(project_id=db427256e2ec4ec084f762a615d0f187)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:36.122469 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3c6a31-8781-4e79-9478-3a8fef78e3f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:36.123727 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4472f215-5368-4a3a-9914-2457d056e733 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:36.123978 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3c6a31-8781-4e79-9478-3a8fef78e3f8 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:36.126215 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 42/86] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:14:35 2022] GET /identity/v3/projects/db427256e2ec4ec084f762a615d0f187 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:36.126968 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4472f215-5368-4a3a-9914-2457d056e733 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:36.226668 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4472f215-5368-4a3a-9914-2457d056e733 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:36.228136 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:36.228484 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 45/87] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:14:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 15:14:36.229334 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3c6a31-8781-4e79-9478-3a8fef78e3f8 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:36.262518 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1686ae9-ba0f-4b6a-a60a-f8376e7c9bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:36.263195 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1686ae9-ba0f-4b6a-a60a-f8376e7c9bec None admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:36.263500 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1686ae9-ba0f-4b6a-a60a-f8376e7c9bec None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:36.263794 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1686ae9-ba0f-4b6a-a60a-f8376e7c9bec None admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:36.264522 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1686ae9-ba0f-4b6a-a60a-f8376e7c9bec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:36.265977 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1686ae9-ba0f-4b6a-a60a-f8376e7c9bec None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:36.273516 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:36.325102 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3c6a31-8781-4e79-9478-3a8fef78e3f8 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:36.356795 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 43/88] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Jul 7 15:14:36 2022] POST /identity/v3/users => generated 289 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:14:36.357437 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1686ae9-ba0f-4b6a-a60a-f8376e7c9bec None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:36.364815 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e4837-c168-4aa5-97bb-0fb5a420899a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:36.365744 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e4837-c168-4aa5-97bb-0fb5a420899a None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:36.366034 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e4837-c168-4aa5-97bb-0fb5a420899a None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:36.366317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e4837-c168-4aa5-97bb-0fb5a420899a None admin] PATH_INFO: `/v3/users/c4236ad3d5d949cf9d36da8484c8bf39` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:36.367745 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1e4837-c168-4aa5-97bb-0fb5a420899a None admin] RBAC: Authorizing `identity:get_user(user_id=c4236ad3d5d949cf9d36da8484c8bf39)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:36.369147 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1e4837-c168-4aa5-97bb-0fb5a420899a None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:36.370901 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 46/89] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:14:36 2022] GET /identity/v3/users/c4236ad3d5d949cf9d36da8484c8bf39 => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:36.371746 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e4837-c168-4aa5-97bb-0fb5a420899a None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:36.482650 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1686ae9-ba0f-4b6a-a60a-f8376e7c9bec None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:36.511031 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e4837-c168-4aa5-97bb-0fb5a420899a None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:36.522008 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf9fdc-ec9f-4480-8a68-d505e741d058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:36.522885 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf9fdc-ec9f-4480-8a68-d505e741d058 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:36.523176 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf9fdc-ec9f-4480-8a68-d505e741d058 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:36.523457 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf9fdc-ec9f-4480-8a68-d505e741d058 None admin] PATH_INFO: `/v3/projects/db427256e2ec4ec084f762a615d0f187` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:36.524827 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf9fdc-ec9f-4480-8a68-d505e741d058 None admin] RBAC: Authorizing `identity:get_project(project_id=db427256e2ec4ec084f762a615d0f187)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:36.526282 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf9fdc-ec9f-4480-8a68-d505e741d058 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:36.526494 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:36.528059 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 44/90] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:14:36 2022] GET /identity/v3/projects/db427256e2ec4ec084f762a615d0f187 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:36.528820 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf9fdc-ec9f-4480-8a68-d505e741d058 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:36.551744 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e04c0c-fa7f-431c-be08-7aed66defded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:36.553598 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e04c0c-fa7f-431c-be08-7aed66defded None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:36.553598 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e04c0c-fa7f-431c-be08-7aed66defded None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:36.553598 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e04c0c-fa7f-431c-be08-7aed66defded None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:36.554379 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e04c0c-fa7f-431c-be08-7aed66defded None admin] RBAC: Adding query filter params (role.id=0e7793fbf3064037984da651879e3a94, scope.project.id=db427256e2ec4ec084f762a615d0f187, user.id=30014035e7d641d5aecffdbe884d3d52) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:36.554697 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e04c0c-fa7f-431c-be08-7aed66defded None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:36.556260 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e04c0c-fa7f-431c-be08-7aed66defded None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:36.558813 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:36.566864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 47/91] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:14:36 2022] GET /identity/v3/role_assignments?user.id=30014035e7d641d5aecffdbe884d3d52&scope.project.id=db427256e2ec4ec084f762a615d0f187&role.id=0e7793fbf3064037984da651879e3a94 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:36.567513 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e04c0c-fa7f-431c-be08-7aed66defded None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:36.660283 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf9fdc-ec9f-4480-8a68-d505e741d058 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:36.704644 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33011b93-a5f0-47eb-b688-ba23e9857b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:36.706508 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33011b93-a5f0-47eb-b688-ba23e9857b56 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:36.707024 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33011b93-a5f0-47eb-b688-ba23e9857b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:36.707429 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33011b93-a5f0-47eb-b688-ba23e9857b56 None admin] PATH_INFO: `/v3/projects/db427256e2ec4ec084f762a615d0f187` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:36.708919 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33011b93-a5f0-47eb-b688-ba23e9857b56 None admin] RBAC: Authorizing `identity:get_project(project_id=db427256e2ec4ec084f762a615d0f187)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:36.710401 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33011b93-a5f0-47eb-b688-ba23e9857b56 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:36.713322 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 45/92] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:14:36 2022] GET /identity/v3/projects/db427256e2ec4ec084f762a615d0f187 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:36.713968 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33011b93-a5f0-47eb-b688-ba23e9857b56 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:36.717019 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e04c0c-fa7f-431c-be08-7aed66defded None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:36.764698 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0daaea-9e55-44b9-a587-a572ac4b326f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:36.765277 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0daaea-9e55-44b9-a587-a572ac4b326f None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:36.765561 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0daaea-9e55-44b9-a587-a572ac4b326f None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:36.765839 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0daaea-9e55-44b9-a587-a572ac4b326f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:36.832142 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:36.847774 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:36.857917 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33011b93-a5f0-47eb-b688-ba23e9857b56 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:36.876050 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0daaea-9e55-44b9-a587-a572ac4b326f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:36.878167 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0daaea-9e55-44b9-a587-a572ac4b326f None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:36.921029 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901eb786-bd19-407b-b6b5-40595b90fc5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:36.921733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901eb786-bd19-407b-b6b5-40595b90fc5d None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:36.922019 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901eb786-bd19-407b-b6b5-40595b90fc5d None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:36.922357 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901eb786-bd19-407b-b6b5-40595b90fc5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:36.923209 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901eb786-bd19-407b-b6b5-40595b90fc5d None admin] RBAC: Adding query filter params (role.id=d43d0d189e254500875f5f86e6abd178, scope.project.id=db427256e2ec4ec084f762a615d0f187, user.id=c4236ad3d5d949cf9d36da8484c8bf39) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:36.923526 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901eb786-bd19-407b-b6b5-40595b90fc5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:36.925574 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901eb786-bd19-407b-b6b5-40595b90fc5d None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:36.934545 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 46/93] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:14:36 2022] GET /identity/v3/role_assignments?user.id=c4236ad3d5d949cf9d36da8484c8bf39&scope.project.id=db427256e2ec4ec084f762a615d0f187&role.id=d43d0d189e254500875f5f86e6abd178 => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:36.938026 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901eb786-bd19-407b-b6b5-40595b90fc5d None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:37.077530 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 48/94] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:14:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 15:14:37.082828 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0daaea-9e55-44b9-a587-a572ac4b326f None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:37.150743 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901eb786-bd19-407b-b6b5-40595b90fc5d None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:37.213516 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22b4fda-b5f1-4888-8632-b3246ba8477a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:37.214247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b4fda-b5f1-4888-8632-b3246ba8477a None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:37.214463 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b4fda-b5f1-4888-8632-b3246ba8477a None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:37.214737 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b4fda-b5f1-4888-8632-b3246ba8477a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:37.215593 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22b4fda-b5f1-4888-8632-b3246ba8477a None admin] RBAC: Adding query filter params (role.id=c82203e4734346719cafb16230f45942, scope.project.id=db427256e2ec4ec084f762a615d0f187, user.id=c4236ad3d5d949cf9d36da8484c8bf39) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:37.215962 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22b4fda-b5f1-4888-8632-b3246ba8477a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:37.217431 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22b4fda-b5f1-4888-8632-b3246ba8477a None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:37.231265 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:37.232663 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 47/95] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:14:36 2022] GET /identity/v3/role_assignments?user.id=c4236ad3d5d949cf9d36da8484c8bf39&scope.project.id=db427256e2ec4ec084f762a615d0f187&role.id=c82203e4734346719cafb16230f45942 => generated 259 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:37.273195 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0daaea-9e55-44b9-a587-a572ac4b326f None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:37.321308 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7066e3b-0b97-4365-898a-a9d71bdd837f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:37.322543 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7066e3b-0b97-4365-898a-a9d71bdd837f None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:37.323020 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7066e3b-0b97-4365-898a-a9d71bdd837f None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:37.323488 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7066e3b-0b97-4365-898a-a9d71bdd837f None admin] PATH_INFO: `/v3/projects/cfaa23d7de8a4cbf96d2831cea78f1b1` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:37.324944 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7066e3b-0b97-4365-898a-a9d71bdd837f None admin] RBAC: Authorizing `identity:get_project(project_id=cfaa23d7de8a4cbf96d2831cea78f1b1)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:37.326718 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7066e3b-0b97-4365-898a-a9d71bdd837f None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:37.329141 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 49/96] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:14:37 2022] GET /identity/v3/projects/cfaa23d7de8a4cbf96d2831cea78f1b1 => generated 312 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:14:37.334653 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22b4fda-b5f1-4888-8632-b3246ba8477a None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:37.370845 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:37.470177 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22b4fda-b5f1-4888-8632-b3246ba8477a None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:37.511415 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247564a6-d0f5-413a-9fe1-f4cbdb5032df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:37.512113 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247564a6-d0f5-413a-9fe1-f4cbdb5032df None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:37.512705 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247564a6-d0f5-413a-9fe1-f4cbdb5032df None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:37.512984 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247564a6-d0f5-413a-9fe1-f4cbdb5032df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:37.513830 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247564a6-d0f5-413a-9fe1-f4cbdb5032df None admin] RBAC: Adding query filter params (role.id=d43d0d189e254500875f5f86e6abd178, scope.project.id=cfaa23d7de8a4cbf96d2831cea78f1b1, user.id=c4236ad3d5d949cf9d36da8484c8bf39) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:37.514145 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247564a6-d0f5-413a-9fe1-f4cbdb5032df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:37.515620 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247564a6-d0f5-413a-9fe1-f4cbdb5032df None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:37.527812 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 48/97] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:14:37 2022] GET /identity/v3/role_assignments?user.id=c4236ad3d5d949cf9d36da8484c8bf39&scope.project.id=cfaa23d7de8a4cbf96d2831cea78f1b1&role.id=d43d0d189e254500875f5f86e6abd178 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:39.176870 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5ed05-6206-4077-9260-b5aafe3cec25 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:39.177859 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5ed05-6206-4077-9260-b5aafe3cec25 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:39.178333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5ed05-6206-4077-9260-b5aafe3cec25 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:39.180142 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 50/98] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:14:39.185528 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae3f7a-3981-452b-a77c-77626272ca47 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:39.186334 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae3f7a-3981-452b-a77c-77626272ca47 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:39.186824 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae3f7a-3981-452b-a77c-77626272ca47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:39.209226 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-5bae3f7a-3981-452b-a77c-77626272ca47 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:39.210696 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-5bae3f7a-3981-452b-a77c-77626272ca47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:39.280267 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 49/99] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:14:39.318940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5ed05-6206-4077-9260-b5aafe3cec25 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:39.451477 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5ed05-6206-4077-9260-b5aafe3cec25 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:39.470382 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa5af0-1542-4ea5-a6d2-69cc2d2966b2 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:39.471016 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa5af0-1542-4ea5-a6d2-69cc2d2966b2 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:39.471487 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa5af0-1542-4ea5-a6d2-69cc2d2966b2 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:39.473362 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 50/100] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:14:39.478221 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b5e6d-2657-4479-9673-48b183ef021b None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:39.478752 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b5e6d-2657-4479-9673-48b183ef021b None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:39.479222 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b5e6d-2657-4479-9673-48b183ef021b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:39.501569 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5068ab22-46ab-466d-8fcc-72007a8684b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:39.503263 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5068ab22-46ab-466d-8fcc-72007a8684b1 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:39.503773 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5068ab22-46ab-466d-8fcc-72007a8684b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:39.504244 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5068ab22-46ab-466d-8fcc-72007a8684b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:39.507439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-e28b5e6d-2657-4479-9673-48b183ef021b None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:39.508949 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-e28b5e6d-2657-4479-9673-48b183ef021b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:39.515487 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5068ab22-46ab-466d-8fcc-72007a8684b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:14:39.523899 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-5068ab22-46ab-466d-8fcc-72007a8684b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:14:39.523899 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:14:39.523899 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:14:39.523899 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:14:39.523899 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:14:39.523899 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:14:39.523899 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:14:39.523899 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:14:39.523899 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:14:39.523899 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:14:39.523899 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:14:39.523899 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:14:39.523899 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:14:39.523899 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:14:39.523899 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:14:39.523899 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:14:39.524765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:14:39.524765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:14:39.524765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:14:39.524765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:14:39.524765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:14:39.524765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:14:39.524765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:14:39.524765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:14:39.524765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:14:39.524765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:14:39.524765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:14:39.524765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:14:39.524765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:14:39.524765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:14:39.524765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:14:39.524765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:14:39.525534 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:14:39.525534 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:14:39.525534 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:14:39.525534 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:14:39.525534 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:14:39.525534 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:14:39.525534 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:14:39.525534 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:14:39.525534 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:14:39.525534 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:14:39.525534 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:14:39.525534 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:14:39.525534 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 51/101] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Jul 7 15:14:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:14:39.532896 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5068ab22-46ab-466d-8fcc-72007a8684b1 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:39.533528 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:39.542722 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:39.639936 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 51/102] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 15:14:39.648686 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5068ab22-46ab-466d-8fcc-72007a8684b1 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:39.677022 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b5e6d-2657-4479-9673-48b183ef021b None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:39.684677 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5b9b6-2459-43f6-bd98-c90d5c5baa7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:39.685406 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5b9b6-2459-43f6-bd98-c90d5c5baa7c None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:39.685687 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5b9b6-2459-43f6-bd98-c90d5c5baa7c None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:39.685961 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5b9b6-2459-43f6-bd98-c90d5c5baa7c None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:39.686763 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b5b9b6-2459-43f6-bd98-c90d5c5baa7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:39.687072 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b5b9b6-2459-43f6-bd98-c90d5c5baa7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:39.688477 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b5b9b6-2459-43f6-bd98-c90d5c5baa7c None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:39.698836 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 52/103] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:14:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:39.707233 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5b9b6-2459-43f6-bd98-c90d5c5baa7c None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:39.781259 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:39.821178 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5b9b6-2459-43f6-bd98-c90d5c5baa7c None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:39.858544 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f93d522-55d6-4b5b-b390-79db1842b911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:39.859684 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93d522-55d6-4b5b-b390-79db1842b911 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:39.860212 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93d522-55d6-4b5b-b390-79db1842b911 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:39.860819 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93d522-55d6-4b5b-b390-79db1842b911 None admin] PATH_INFO: `/v3/users/30014035e7d641d5aecffdbe884d3d52` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:39.862848 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f93d522-55d6-4b5b-b390-79db1842b911 None admin] RBAC: Authorizing `identity:get_user(user_id=30014035e7d641d5aecffdbe884d3d52)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:39.865669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f93d522-55d6-4b5b-b390-79db1842b911 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:39.866363 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 53/104] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:14:39 2022] GET /identity/v3/users/30014035e7d641d5aecffdbe884d3d52 => generated 247 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:14:39.867795 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b359f-0e63-4a73-b7c3-dbee0a8f4689 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:39.868096 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b359f-0e63-4a73-b7c3-dbee0a8f4689 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:39.868371 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b359f-0e63-4a73-b7c3-dbee0a8f4689 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:39.869853 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 54/105] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:14:39.874464 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b359f-0e63-4a73-b7c3-dbee0a8f4689 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:39.895308 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b5e6d-2657-4479-9673-48b183ef021b None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:39.940923 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8afd9da-e6bb-4f82-bc06-7ba47ce20460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:39.941882 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afd9da-e6bb-4f82-bc06-7ba47ce20460 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:39.942360 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afd9da-e6bb-4f82-bc06-7ba47ce20460 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:39.942783 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afd9da-e6bb-4f82-bc06-7ba47ce20460 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:39.944386 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8afd9da-e6bb-4f82-bc06-7ba47ce20460 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:39.945993 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8afd9da-e6bb-4f82-bc06-7ba47ce20460 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:39.948409 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 52/106] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:14:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:39.949673 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaacd4b-eae6-42db-adb2-853d17491cc1 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:39.950106 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaacd4b-eae6-42db-adb2-853d17491cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:39.950590 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaacd4b-eae6-42db-adb2-853d17491cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:39.984147 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-ccaacd4b-eae6-42db-adb2-853d17491cc1 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:39.985423 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-ccaacd4b-eae6-42db-adb2-853d17491cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:39.995772 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:40.008363 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b359f-0e63-4a73-b7c3-dbee0a8f4689 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:40.046130 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3756575-0dd3-4e59-8357-88ebd40bf661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:40.047228 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3756575-0dd3-4e59-8357-88ebd40bf661 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:40.047788 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3756575-0dd3-4e59-8357-88ebd40bf661 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:40.048233 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3756575-0dd3-4e59-8357-88ebd40bf661 None admin] PATH_INFO: `/v3/projects/db427256e2ec4ec084f762a615d0f187` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:40.050306 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3756575-0dd3-4e59-8357-88ebd40bf661 None admin] RBAC: Authorizing `identity:get_project(project_id=db427256e2ec4ec084f762a615d0f187)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:40.052251 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3756575-0dd3-4e59-8357-88ebd40bf661 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:40.056596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 53/108] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:14:40.058144 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73843423-528b-429f-9d12-c522c3afe60e None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:40.058664 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73843423-528b-429f-9d12-c522c3afe60e None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:40.058987 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 55/108] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:14:39 2022] GET /identity/v3/projects/db427256e2ec4ec084f762a615d0f187 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:40.059290 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73843423-528b-429f-9d12-c522c3afe60e None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:40.060157 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3756575-0dd3-4e59-8357-88ebd40bf661 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:40.060899 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 54/109] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:14:40.061543 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73843423-528b-429f-9d12-c522c3afe60e None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:40.063728 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:40.168113 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73843423-528b-429f-9d12-c522c3afe60e None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:40.172136 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3756575-0dd3-4e59-8357-88ebd40bf661 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:40.204075 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00199291-4a9b-4bd5-8470-197668d43015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:40.204973 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00199291-4a9b-4bd5-8470-197668d43015 None admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:40.205286 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00199291-4a9b-4bd5-8470-197668d43015 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:40.205562 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00199291-4a9b-4bd5-8470-197668d43015 None admin] PATH_INFO: `/v3/projects/db427256e2ec4ec084f762a615d0f187/users/30014035e7d641d5aecffdbe884d3d52/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:40.207534 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00199291-4a9b-4bd5-8470-197668d43015 None admin] RBAC: Authorizing `identity:create_grant(project_id=db427256e2ec4ec084f762a615d0f187, user_id=30014035e7d641d5aecffdbe884d3d52, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:40.209014 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00199291-4a9b-4bd5-8470-197668d43015 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:40.209907 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f5d75-f028-4d71-a17e-bed66776235e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:40.211199 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f5d75-f028-4d71-a17e-bed66776235e None admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:40.211698 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f5d75-f028-4d71-a17e-bed66776235e None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:40.212161 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f5d75-f028-4d71-a17e-bed66776235e None admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:40.213017 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f5d75-f028-4d71-a17e-bed66776235e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:40.214446 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f5d75-f028-4d71-a17e-bed66776235e None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:40.279221 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 55/110] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Jul 7 15:14:40 2022] PUT /identity/v3/projects/db427256e2ec4ec084f762a615d0f187/users/30014035e7d641d5aecffdbe884d3d52/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:14:40.280721 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5032639-5c9e-45fa-9fc7-a990dec74615 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:40.281007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5032639-5c9e-45fa-9fc7-a990dec74615 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:40.281283 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5032639-5c9e-45fa-9fc7-a990dec74615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:40.288010 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:40.302773 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-b5032639-5c9e-45fa-9fc7-a990dec74615 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:40.304217 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-b5032639-5c9e-45fa-9fc7-a990dec74615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:40.308171 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 56/111] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Jul 7 15:14:40 2022] POST /identity/v3/projects => generated 302 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:14:40.310386 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f5d75-f028-4d71-a17e-bed66776235e None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:40.393247 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:40.459453 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f5d75-f028-4d71-a17e-bed66776235e None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:40.492048 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 56/112] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:14:40.494444 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60b1d6-d4bd-4638-91de-f53cc0c9861e None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:40.495450 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60b1d6-d4bd-4638-91de-f53cc0c9861e None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:40.496218 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60b1d6-d4bd-4638-91de-f53cc0c9861e None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:40.498809 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 57/113] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:14:40.505173 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405af931-da81-4844-a58e-f068c977a36b None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:40.505890 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405af931-da81-4844-a58e-f068c977a36b None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:40.506470 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405af931-da81-4844-a58e-f068c977a36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:40.511722 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67b3fbf-0330-41fe-996c-3ceeb787929b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:40.512487 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b3fbf-0330-41fe-996c-3ceeb787929b None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:40.512777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b3fbf-0330-41fe-996c-3ceeb787929b None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:40.513105 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b3fbf-0330-41fe-996c-3ceeb787929b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:40.526144 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67b3fbf-0330-41fe-996c-3ceeb787929b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:14:40.535311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-e67b3fbf-0330-41fe-996c-3ceeb787929b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 07 15:14:40.535311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:14:40.535311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:14:40.535311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:14:40.535311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:14:40.535311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:14:40.535311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:14:40.535311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:14:40.535311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:14:40.535311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:14:40.535311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:14:40.535311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:14:40.535311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:14:40.535311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:14:40.535311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:14:40.535311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:14:40.536260 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:14:40.536260 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:14:40.536260 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:14:40.536260 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:14:40.536260 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:14:40.536260 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:14:40.536260 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:14:40.536260 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:14:40.536260 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:14:40.536260 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:14:40.536260 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:14:40.536260 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:14:40.536260 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:14:40.536260 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:14:40.536260 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:14:40.536260 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:14:40.537067 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:14:40.537067 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:14:40.537067 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:14:40.537067 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:14:40.537067 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:14:40.537067 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:14:40.537067 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:14:40.537067 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:14:40.537067 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:14:40.537067 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:14:40.537067 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 07 15:14:40.537067 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:14:40.537067 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 57/114] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:14:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:14:40.537748 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67b3fbf-0330-41fe-996c-3ceeb787929b None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:40.560678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-405af931-da81-4844-a58e-f068c977a36b None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:40.561894 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-405af931-da81-4844-a58e-f068c977a36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:40.650373 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 58/115] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:14:40.652268 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405af931-da81-4844-a58e-f068c977a36b None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:40.698732 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67b3fbf-0330-41fe-996c-3ceeb787929b None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:40.737206 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:40.743820 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:40.777880 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16730e2e-9a37-423e-a6cf-c2cdbd8981b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:40.778593 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16730e2e-9a37-423e-a6cf-c2cdbd8981b5 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:40.778878 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16730e2e-9a37-423e-a6cf-c2cdbd8981b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:40.779231 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16730e2e-9a37-423e-a6cf-c2cdbd8981b5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:40.784991 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405af931-da81-4844-a58e-f068c977a36b None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:40.797098 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16730e2e-9a37-423e-a6cf-c2cdbd8981b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:14:40.820034 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-16730e2e-9a37-423e-a6cf-c2cdbd8981b5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 07 15:14:40.820034 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:14:40.820034 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:14:40.820034 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:14:40.820034 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:14:40.820034 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:14:40.820034 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:14:40.820034 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:14:40.820034 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:14:40.820034 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:14:40.820034 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:14:40.820034 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:14:40.820034 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:14:40.820034 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:14:40.820034 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:14:40.820034 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:14:40.821044 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:14:40.821044 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:14:40.821044 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:14:40.821044 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:14:40.821044 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:14:40.821044 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:14:40.821044 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:14:40.821044 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:14:40.821044 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:14:40.821044 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:14:40.821044 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:14:40.821044 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:14:40.821044 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:14:40.821044 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:14:40.821044 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:14:40.821044 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:14:40.822608 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:14:40.822608 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:14:40.822608 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:14:40.822608 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:14:40.822608 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:14:40.822608 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:14:40.822608 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:14:40.822608 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:14:40.822608 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:14:40.822608 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:14:40.822608 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 07 15:14:40.822608 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:14:40.822608 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 58/116] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:14:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:14:40.823370 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16730e2e-9a37-423e-a6cf-c2cdbd8981b5 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:40.836880 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63ca78-489f-4d07-b98e-8e586050ce20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:40.837748 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63ca78-489f-4d07-b98e-8e586050ce20 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:40.838219 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63ca78-489f-4d07-b98e-8e586050ce20 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:40.838598 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63ca78-489f-4d07-b98e-8e586050ce20 None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:40.839476 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b63ca78-489f-4d07-b98e-8e586050ce20 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:40.839936 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b63ca78-489f-4d07-b98e-8e586050ce20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:40.841653 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b63ca78-489f-4d07-b98e-8e586050ce20 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:40.867627 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 59/117] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Jul 7 15:14:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:40.868928 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63ca78-489f-4d07-b98e-8e586050ce20 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:40.946983 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16730e2e-9a37-423e-a6cf-c2cdbd8981b5 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:40.994179 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63ca78-489f-4d07-b98e-8e586050ce20 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:41.003670 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b16028-7e10-4e22-a058-79e70f40e15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:41.005225 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b16028-7e10-4e22-a058-79e70f40e15b None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:41.005690 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b16028-7e10-4e22-a058-79e70f40e15b None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:41.006544 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b16028-7e10-4e22-a058-79e70f40e15b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:41.020689 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b16028-7e10-4e22-a058-79e70f40e15b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:14:41.024912 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:41.033669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-41b16028-7e10-4e22-a058-79e70f40e15b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 15:14:41.033669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:14:41.033669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:14:41.033669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:14:41.033669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:14:41.033669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:14:41.033669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:14:41.033669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:14:41.033669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:14:41.033669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:14:41.033669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:14:41.033669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:14:41.033669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:14:41.033669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:14:41.033669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:14:41.033669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:14:41.034607 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:14:41.034607 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:14:41.034607 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:14:41.034607 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:14:41.034607 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:14:41.034607 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:14:41.034607 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:14:41.034607 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:14:41.034607 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:14:41.034607 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:14:41.034607 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:14:41.034607 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:14:41.034607 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:14:41.034607 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:14:41.034607 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:14:41.034607 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:14:41.035469 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:14:41.035469 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:14:41.035469 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:14:41.035469 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:14:41.035469 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:14:41.035469 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:14:41.035469 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:14:41.035469 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:14:41.035469 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:14:41.035469 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:14:41.035469 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 15:14:41.035469 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:14:41.036258 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddbbf4d-5e48-4494-9acc-48fe631989d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:41.036258 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbbf4d-5e48-4494-9acc-48fe631989d9 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:41.036258 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbbf4d-5e48-4494-9acc-48fe631989d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:41.036258 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbbf4d-5e48-4494-9acc-48fe631989d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:41.036954 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddbbf4d-5e48-4494-9acc-48fe631989d9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:41.037564 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddbbf4d-5e48-4494-9acc-48fe631989d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:41.039027 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddbbf4d-5e48-4494-9acc-48fe631989d9 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:41.039480 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 59/118] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Jul 7 15:14:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:14:41.041239 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca212a4-638d-4c58-bfdc-b1d81b3b3444 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:41.042119 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca212a4-638d-4c58-bfdc-b1d81b3b3444 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:41.042546 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca212a4-638d-4c58-bfdc-b1d81b3b3444 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:41.044033 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 60/119] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:14:41.067318 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca212a4-638d-4c58-bfdc-b1d81b3b3444 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:41.067461 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 60/120] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Jul 7 15:14:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:41.067461 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddbbf4d-5e48-4494-9acc-48fe631989d9 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:41.091235 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:41.164167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddbbf4d-5e48-4494-9acc-48fe631989d9 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:41.198980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca212a4-638d-4c58-bfdc-b1d81b3b3444 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:41.220233 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca97cd98-d218-40fa-976d-2d8ee41291a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:41.221665 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97cd98-d218-40fa-976d-2d8ee41291a5 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:41.222133 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97cd98-d218-40fa-976d-2d8ee41291a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:41.223037 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97cd98-d218-40fa-976d-2d8ee41291a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:41.224071 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca97cd98-d218-40fa-976d-2d8ee41291a5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:41.224570 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca97cd98-d218-40fa-976d-2d8ee41291a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:41.226397 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca97cd98-d218-40fa-976d-2d8ee41291a5 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:41.236295 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 61/121] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:14:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:41.238418 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a883f-6b31-480b-961b-9cbfd7598d1a None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:41.238935 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a883f-6b31-480b-961b-9cbfd7598d1a None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:41.239441 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a883f-6b31-480b-961b-9cbfd7598d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:41.244353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc2a263-6558-47e0-b23e-caa524a0c256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:41.245176 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2a263-6558-47e0-b23e-caa524a0c256 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:41.245462 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2a263-6558-47e0-b23e-caa524a0c256 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:41.245743 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2a263-6558-47e0-b23e-caa524a0c256 None admin] PATH_INFO: `/v3/users/95a4c061562d43ad91c3d9188ae2c869` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:41.259057 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-d52a883f-6b31-480b-961b-9cbfd7598d1a None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:41.260641 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-d52a883f-6b31-480b-961b-9cbfd7598d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:41.264708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc2a263-6558-47e0-b23e-caa524a0c256 None admin] RBAC: Authorizing `identity:get_user(user_id=95a4c061562d43ad91c3d9188ae2c869)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:41.266168 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc2a263-6558-47e0-b23e-caa524a0c256 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:41.272568 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 61/122] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:14:41 2022] GET /identity/v3/users/95a4c061562d43ad91c3d9188ae2c869 => generated 289 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:41.273402 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc2a263-6558-47e0-b23e-caa524a0c256 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:41.332415 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:41.355357 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 62/123] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:14:41.356362 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a883f-6b31-480b-961b-9cbfd7598d1a None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:41.388228 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:41.398035 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc2a263-6558-47e0-b23e-caa524a0c256 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:41.436231 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec747d7-0ba6-4b83-b4ee-3d6b1a978990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:41.437244 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec747d7-0ba6-4b83-b4ee-3d6b1a978990 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:41.437951 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec747d7-0ba6-4b83-b4ee-3d6b1a978990 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:41.438067 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec747d7-0ba6-4b83-b4ee-3d6b1a978990 None admin] PATH_INFO: `/v3/users/c4236ad3d5d949cf9d36da8484c8bf39` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:41.440269 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec747d7-0ba6-4b83-b4ee-3d6b1a978990 None admin] RBAC: Authorizing `identity:get_user(user_id=c4236ad3d5d949cf9d36da8484c8bf39)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:41.441819 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec747d7-0ba6-4b83-b4ee-3d6b1a978990 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:41.444069 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 62/124] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:14:41 2022] GET /identity/v3/users/c4236ad3d5d949cf9d36da8484c8bf39 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:41.445044 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec747d7-0ba6-4b83-b4ee-3d6b1a978990 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:41.462538 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a883f-6b31-480b-961b-9cbfd7598d1a None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:41.500270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda2b2ab-a2ca-436e-8530-0300955898ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:41.501476 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2b2ab-a2ca-436e-8530-0300955898ea None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:41.501905 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2b2ab-a2ca-436e-8530-0300955898ea None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:41.502884 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2b2ab-a2ca-436e-8530-0300955898ea None admin] PATH_INFO: `/v3/users/c4236ad3d5d949cf9d36da8484c8bf39` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:41.504477 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda2b2ab-a2ca-436e-8530-0300955898ea None admin] RBAC: Authorizing `identity:get_user(user_id=c4236ad3d5d949cf9d36da8484c8bf39)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:41.506110 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda2b2ab-a2ca-436e-8530-0300955898ea None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:41.507946 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 63/125] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:14:41 2022] GET /identity/v3/users/c4236ad3d5d949cf9d36da8484c8bf39 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:41.509034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda2b2ab-a2ca-436e-8530-0300955898ea None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:41.564384 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec747d7-0ba6-4b83-b4ee-3d6b1a978990 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:41.566328 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:41.603034 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2404ee37-db8d-4e9c-ab6c-81a72b3ac98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:41.603891 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404ee37-db8d-4e9c-ab6c-81a72b3ac98c None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:41.604376 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404ee37-db8d-4e9c-ab6c-81a72b3ac98c None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:41.604675 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404ee37-db8d-4e9c-ab6c-81a72b3ac98c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:41.620822 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda2b2ab-a2ca-436e-8530-0300955898ea None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:41.634775 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:41.654550 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9018091c-a83d-4bbe-b8fd-24ee4a2dc4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:41.655539 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018091c-a83d-4bbe-b8fd-24ee4a2dc4fb None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:41.655983 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018091c-a83d-4bbe-b8fd-24ee4a2dc4fb None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:41.656424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018091c-a83d-4bbe-b8fd-24ee4a2dc4fb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:41.667644 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9018091c-a83d-4bbe-b8fd-24ee4a2dc4fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:14:41.677107 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-9018091c-a83d-4bbe-b8fd-24ee4a2dc4fb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 07 15:14:41.677107 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:14:41.677107 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:14:41.677107 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:14:41.677107 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:14:41.677107 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:14:41.677107 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:14:41.677107 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:14:41.677107 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:14:41.677107 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:14:41.677107 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:14:41.677107 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:14:41.677107 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:14:41.677107 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:14:41.677107 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:14:41.677107 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:14:41.678302 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:14:41.678302 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:14:41.678302 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:14:41.678302 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:14:41.678302 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:14:41.678302 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:14:41.678302 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:14:41.678302 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:14:41.678302 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:14:41.678302 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:14:41.678302 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:14:41.678302 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:14:41.678302 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:14:41.678302 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:14:41.678302 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:14:41.678302 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:14:41.679270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:14:41.679270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:14:41.679270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:14:41.679270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:14:41.679270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:14:41.679270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:14:41.679270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:14:41.679270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:14:41.679270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:14:41.679270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:14:41.679270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 07 15:14:41.679270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:14:41.681442 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2404ee37-db8d-4e9c-ab6c-81a72b3ac98c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:41.681853 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 64/126] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:14:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:14:41.682865 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9018091c-a83d-4bbe-b8fd-24ee4a2dc4fb None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:41.683378 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2404ee37-db8d-4e9c-ab6c-81a72b3ac98c None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:41.792328 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9018091c-a83d-4bbe-b8fd-24ee4a2dc4fb None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:41.796917 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 63/127] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:14:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 15:14:41.796917 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2404ee37-db8d-4e9c-ab6c-81a72b3ac98c None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:41.807936 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:41.834770 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22151edf-13db-448e-815c-a6268679e0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:41.835990 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22151edf-13db-448e-815c-a6268679e0a7 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:41.836480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22151edf-13db-448e-815c-a6268679e0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:41.836934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22151edf-13db-448e-815c-a6268679e0a7 None admin] PATH_INFO: `/v3/projects/db427256e2ec4ec084f762a615d0f187` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:41.838386 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22151edf-13db-448e-815c-a6268679e0a7 None admin] RBAC: Authorizing `identity:get_project(project_id=db427256e2ec4ec084f762a615d0f187)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:41.840190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22151edf-13db-448e-815c-a6268679e0a7 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:41.842337 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 65/128] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:14:41 2022] GET /identity/v3/projects/db427256e2ec4ec084f762a615d0f187 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:41.843497 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22151edf-13db-448e-815c-a6268679e0a7 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:41.861236 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:41.910763 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2404ee37-db8d-4e9c-ab6c-81a72b3ac98c None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:41.948115 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bbb3e6-37f1-4ece-9b88-839752b8165f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:41.949311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbb3e6-37f1-4ece-9b88-839752b8165f None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:41.949818 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbb3e6-37f1-4ece-9b88-839752b8165f None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:41.950284 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbb3e6-37f1-4ece-9b88-839752b8165f None admin] PATH_INFO: `/v3/projects/db427256e2ec4ec084f762a615d0f187` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:41.951755 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bbb3e6-37f1-4ece-9b88-839752b8165f None admin] RBAC: Authorizing `identity:get_project(project_id=db427256e2ec4ec084f762a615d0f187)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:41.953541 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bbb3e6-37f1-4ece-9b88-839752b8165f None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:41.956100 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 64/129] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:14:41 2022] GET /identity/v3/projects/db427256e2ec4ec084f762a615d0f187 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:41.957312 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bbb3e6-37f1-4ece-9b88-839752b8165f None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:41.995116 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22151edf-13db-448e-815c-a6268679e0a7 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:42.039669 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a9781d-51f5-4100-aab4-4f3dc0f8f544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:42.040345 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9781d-51f5-4100-aab4-4f3dc0f8f544 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:42.040636 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9781d-51f5-4100-aab4-4f3dc0f8f544 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:42.040919 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9781d-51f5-4100-aab4-4f3dc0f8f544 None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:42.041688 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a9781d-51f5-4100-aab4-4f3dc0f8f544 None admin] RBAC: Adding query filter params (name=member) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:42.042061 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a9781d-51f5-4100-aab4-4f3dc0f8f544 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:42.043516 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a9781d-51f5-4100-aab4-4f3dc0f8f544 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:42.054743 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 66/130] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Jul 7 15:14:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:42.056900 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a9781d-51f5-4100-aab4-4f3dc0f8f544 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:42.075485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bbb3e6-37f1-4ece-9b88-839752b8165f None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:42.104948 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:42.112983 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:42.113603 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaf716b-6184-4f61-b2b8-252ef894fea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:42.114586 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf716b-6184-4f61-b2b8-252ef894fea9 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:42.114869 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf716b-6184-4f61-b2b8-252ef894fea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:42.115213 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf716b-6184-4f61-b2b8-252ef894fea9 None admin] PATH_INFO: `/v3/projects/db427256e2ec4ec084f762a615d0f187` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:42.116429 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaf716b-6184-4f61-b2b8-252ef894fea9 None admin] RBAC: Authorizing `identity:get_project(project_id=db427256e2ec4ec084f762a615d0f187)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:42.117877 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaf716b-6184-4f61-b2b8-252ef894fea9 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:42.119933 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 65/131] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:14:41 2022] GET /identity/v3/projects/db427256e2ec4ec084f762a615d0f187 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:42.124405 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaf716b-6184-4f61-b2b8-252ef894fea9 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:42.196044 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a9781d-51f5-4100-aab4-4f3dc0f8f544 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:42.236100 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e474acc-148a-4dbf-9c1c-9ed7bec790e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:42.238466 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e474acc-148a-4dbf-9c1c-9ed7bec790e1 None admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:42.238978 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e474acc-148a-4dbf-9c1c-9ed7bec790e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:42.239554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e474acc-148a-4dbf-9c1c-9ed7bec790e1 None admin] PATH_INFO: `/v3/projects/db427256e2ec4ec084f762a615d0f187/users/c4236ad3d5d949cf9d36da8484c8bf39/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:42.241925 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e474acc-148a-4dbf-9c1c-9ed7bec790e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=db427256e2ec4ec084f762a615d0f187, user_id=c4236ad3d5d949cf9d36da8484c8bf39, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:42.243291 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaf716b-6184-4f61-b2b8-252ef894fea9 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:42.243887 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e474acc-148a-4dbf-9c1c-9ed7bec790e1 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:42.280258 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6606686b-e247-4d52-989c-c98b8c5a39f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:42.281197 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606686b-e247-4d52-989c-c98b8c5a39f5 None admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:42.281486 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606686b-e247-4d52-989c-c98b8c5a39f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:42.281759 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606686b-e247-4d52-989c-c98b8c5a39f5 None admin] PATH_INFO: `/v3/projects/db427256e2ec4ec084f762a615d0f187/users/c4236ad3d5d949cf9d36da8484c8bf39/roles/c82203e4734346719cafb16230f45942` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:42.284207 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6606686b-e247-4d52-989c-c98b8c5a39f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=db427256e2ec4ec084f762a615d0f187, user_id=c4236ad3d5d949cf9d36da8484c8bf39, role_id=c82203e4734346719cafb16230f45942)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:42.285734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6606686b-e247-4d52-989c-c98b8c5a39f5 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:42.347832 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 67/132] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Jul 7 15:14:42 2022] PUT /identity/v3/projects/db427256e2ec4ec084f762a615d0f187/users/c4236ad3d5d949cf9d36da8484c8bf39/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:14:42.347832 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e474acc-148a-4dbf-9c1c-9ed7bec790e1 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:42.370734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 66/133] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Jul 7 15:14:42 2022] PUT /identity/v3/projects/db427256e2ec4ec084f762a615d0f187/users/c4236ad3d5d949cf9d36da8484c8bf39/roles/c82203e4734346719cafb16230f45942 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:14:42.371867 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6606686b-e247-4d52-989c-c98b8c5a39f5 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:42.488350 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:42.546040 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6606686b-e247-4d52-989c-c98b8c5a39f5 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:42.559481 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:42.572998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e474acc-148a-4dbf-9c1c-9ed7bec790e1 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:42.593086 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762332e6-151f-4569-a819-731dc9a278c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:42.599004 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762332e6-151f-4569-a819-731dc9a278c8 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:42.599004 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762332e6-151f-4569-a819-731dc9a278c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:42.599004 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762332e6-151f-4569-a819-731dc9a278c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:42.599004 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762332e6-151f-4569-a819-731dc9a278c8 None admin] RBAC: Adding query filter params (role.id=4fe04bd6b66542379945fab86d689ae6, scope.project.id=db427256e2ec4ec084f762a615d0f187, user.id=95a4c061562d43ad91c3d9188ae2c869) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:42.599004 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762332e6-151f-4569-a819-731dc9a278c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:42.599004 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762332e6-151f-4569-a819-731dc9a278c8 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:42.605572 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 67/134] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:14:42 2022] GET /identity/v3/role_assignments?user.id=95a4c061562d43ad91c3d9188ae2c869&scope.project.id=db427256e2ec4ec084f762a615d0f187&role.id=4fe04bd6b66542379945fab86d689ae6 => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:42.607283 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a3337-0267-41a9-b653-6da6abab4575 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:42.607846 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a3337-0267-41a9-b653-6da6abab4575 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:42.608354 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a3337-0267-41a9-b653-6da6abab4575 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:42.609945 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 68/135] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:14:42.619719 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c50db-62ae-4e13-9c39-db79f2e65a5d None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:42.619719 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c50db-62ae-4e13-9c39-db79f2e65a5d None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:42.619719 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c50db-62ae-4e13-9c39-db79f2e65a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:42.626431 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd5eda-57ea-4b7d-8637-e86aaac7ff16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:42.627434 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd5eda-57ea-4b7d-8637-e86aaac7ff16 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:42.627746 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd5eda-57ea-4b7d-8637-e86aaac7ff16 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:42.628020 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd5eda-57ea-4b7d-8637-e86aaac7ff16 None admin] PATH_INFO: `/v3/users/c4236ad3d5d949cf9d36da8484c8bf39` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:42.632322 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afd5eda-57ea-4b7d-8637-e86aaac7ff16 None admin] RBAC: Authorizing `identity:get_user(user_id=c4236ad3d5d949cf9d36da8484c8bf39)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:42.633838 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afd5eda-57ea-4b7d-8637-e86aaac7ff16 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:42.640139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 68/136] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:14:42 2022] GET /identity/v3/users/c4236ad3d5d949cf9d36da8484c8bf39 => generated 275 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:42.641205 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd5eda-57ea-4b7d-8637-e86aaac7ff16 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:42.645676 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-aa5c50db-62ae-4e13-9c39-db79f2e65a5d None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:42.648241 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-aa5c50db-62ae-4e13-9c39-db79f2e65a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:42.793841 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 69/137] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 15:14:42.832407 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5c50db-62ae-4e13-9c39-db79f2e65a5d None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:42.873847 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd5eda-57ea-4b7d-8637-e86aaac7ff16 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:42.930969 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd87b101-e710-4e5f-8ee9-4ec8dfbeaa61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:42.932260 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87b101-e710-4e5f-8ee9-4ec8dfbeaa61 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:42.932551 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87b101-e710-4e5f-8ee9-4ec8dfbeaa61 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:42.932933 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87b101-e710-4e5f-8ee9-4ec8dfbeaa61 None admin] PATH_INFO: `/v3/projects/cfaa23d7de8a4cbf96d2831cea78f1b1` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:42.934151 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd87b101-e710-4e5f-8ee9-4ec8dfbeaa61 None admin] RBAC: Authorizing `identity:get_project(project_id=cfaa23d7de8a4cbf96d2831cea78f1b1)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:42.935650 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd87b101-e710-4e5f-8ee9-4ec8dfbeaa61 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:42.937688 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 69/138] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:14:42 2022] GET /identity/v3/projects/cfaa23d7de8a4cbf96d2831cea78f1b1 => generated 312 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:42.939029 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a657f55-9c8f-43ef-a195-a1ff316fa959 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:42.939320 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a657f55-9c8f-43ef-a195-a1ff316fa959 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:42.939656 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a657f55-9c8f-43ef-a195-a1ff316fa959 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:42.940872 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 70/139] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:14:42.945675 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a657f55-9c8f-43ef-a195-a1ff316fa959 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:42.948746 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:43.004030 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:43.060153 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a657f55-9c8f-43ef-a195-a1ff316fa959 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:43.094872 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3237e4-ead2-4d2b-bd2a-59485a7d51ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:43.096520 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3237e4-ead2-4d2b-bd2a-59485a7d51ba None admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:43.096807 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3237e4-ead2-4d2b-bd2a-59485a7d51ba None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:43.097082 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3237e4-ead2-4d2b-bd2a-59485a7d51ba None admin] PATH_INFO: `/v3/projects/cfaa23d7de8a4cbf96d2831cea78f1b1/users/c4236ad3d5d949cf9d36da8484c8bf39/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:43.098684 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5c50db-62ae-4e13-9c39-db79f2e65a5d None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:43.099309 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3237e4-ead2-4d2b-bd2a-59485a7d51ba None admin] RBAC: Authorizing `identity:create_grant(project_id=cfaa23d7de8a4cbf96d2831cea78f1b1, user_id=c4236ad3d5d949cf9d36da8484c8bf39, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:43.100873 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3237e4-ead2-4d2b-bd2a-59485a7d51ba None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:43.135159 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da7ee0-e9a9-4fc3-930c-8d1002850b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:43.135916 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da7ee0-e9a9-4fc3-930c-8d1002850b89 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:43.136208 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da7ee0-e9a9-4fc3-930c-8d1002850b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:43.136483 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da7ee0-e9a9-4fc3-930c-8d1002850b89 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:43.149621 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da7ee0-e9a9-4fc3-930c-8d1002850b89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:14:43.157366 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-c6da7ee0-e9a9-4fc3-930c-8d1002850b89 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:14:43.157366 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:14:43.157366 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:14:43.157366 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:14:43.157366 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:14:43.157366 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:14:43.157366 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:14:43.157366 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:14:43.157366 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:14:43.157366 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:14:43.157366 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:14:43.157366 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:14:43.157366 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:14:43.157366 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:14:43.157366 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:14:43.157366 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:14:43.163471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:14:43.163471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:14:43.163471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:14:43.163471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:14:43.163471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:14:43.163471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:14:43.163471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:14:43.163471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:14:43.163471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:14:43.163471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:14:43.163471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:14:43.163471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:14:43.163471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:14:43.163471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:14:43.163471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:14:43.163471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:14:43.164428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:14:43.164428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:14:43.164428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:14:43.164428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:14:43.164428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:14:43.164428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:14:43.164428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:14:43.164428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:14:43.164428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:14:43.164428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:14:43.164428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:14:43.164428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:14:43.164428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 70/140] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Jul 7 15:14:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:14:43.164428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0176d6-c667-434b-b40d-dd0b454b0bf5 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:43.165270 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0176d6-c667-434b-b40d-dd0b454b0bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:43.165270 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0176d6-c667-434b-b40d-dd0b454b0bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:43.177995 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-1b0176d6-c667-434b-b40d-dd0b454b0bf5 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:43.179201 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-1b0176d6-c667-434b-b40d-dd0b454b0bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:43.199304 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 71/141] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Jul 7 15:14:42 2022] PUT /identity/v3/projects/cfaa23d7de8a4cbf96d2831cea78f1b1/users/c4236ad3d5d949cf9d36da8484c8bf39/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:14:43.202563 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3237e4-ead2-4d2b-bd2a-59485a7d51ba None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:43.293595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 71/142] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:14:43.339861 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0176d6-c667-434b-b40d-dd0b454b0bf5 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:43.400787 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:43.413628 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:43.416164 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3237e4-ead2-4d2b-bd2a-59485a7d51ba None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:43.474114 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0176d6-c667-434b-b40d-dd0b454b0bf5 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:43.496116 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7847cb2-4089-48ec-aaab-d375d6cfca7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:43.497356 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7847cb2-4089-48ec-aaab-d375d6cfca7e None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:43.497844 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7847cb2-4089-48ec-aaab-d375d6cfca7e None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:43.498327 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7847cb2-4089-48ec-aaab-d375d6cfca7e None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:43.499293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7847cb2-4089-48ec-aaab-d375d6cfca7e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:43.500481 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7847cb2-4089-48ec-aaab-d375d6cfca7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:43.502778 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7847cb2-4089-48ec-aaab-d375d6cfca7e None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:43.511361 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa481e01-6a63-4fff-b719-3d4a84b16065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:43.512222 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa481e01-6a63-4fff-b719-3d4a84b16065 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:43.512543 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa481e01-6a63-4fff-b719-3d4a84b16065 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:43.512828 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa481e01-6a63-4fff-b719-3d4a84b16065 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:43.514067 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa481e01-6a63-4fff-b719-3d4a84b16065 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:43.515509 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa481e01-6a63-4fff-b719-3d4a84b16065 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:43.517184 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 72/143] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:14:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:14:43.522659 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 72/144] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:14:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:43.522929 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa481e01-6a63-4fff-b719-3d4a84b16065 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:43.528066 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7847cb2-4089-48ec-aaab-d375d6cfca7e None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:43.628789 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa481e01-6a63-4fff-b719-3d4a84b16065 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:43.639909 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:43.663758 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c904cb-5de4-4239-9c7d-c6e9dc381790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:43.664362 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c904cb-5de4-4239-9c7d-c6e9dc381790 None admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:43.664651 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c904cb-5de4-4239-9c7d-c6e9dc381790 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:43.664938 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c904cb-5de4-4239-9c7d-c6e9dc381790 None admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:43.665668 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c904cb-5de4-4239-9c7d-c6e9dc381790 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:43.667118 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c904cb-5de4-4239-9c7d-c6e9dc381790 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:43.695634 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7847cb2-4089-48ec-aaab-d375d6cfca7e None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:43.730349 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec5f04-5f88-442a-bcdb-7a06971e6f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:43.731224 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec5f04-5f88-442a-bcdb-7a06971e6f62 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:43.731509 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec5f04-5f88-442a-bcdb-7a06971e6f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:43.731831 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec5f04-5f88-442a-bcdb-7a06971e6f62 None admin] PATH_INFO: `/v3/users/30014035e7d641d5aecffdbe884d3d52` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:43.733035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec5f04-5f88-442a-bcdb-7a06971e6f62 None admin] RBAC: Authorizing `identity:get_user(user_id=30014035e7d641d5aecffdbe884d3d52)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:43.734480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec5f04-5f88-442a-bcdb-7a06971e6f62 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:43.737098 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 73/145] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:14:43 2022] GET /identity/v3/users/30014035e7d641d5aecffdbe884d3d52 => generated 247 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:43.742164 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec5f04-5f88-442a-bcdb-7a06971e6f62 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:43.768434 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:43.797466 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 73/146] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Jul 7 15:14:43 2022] POST /identity/v3/users => generated 283 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:14:43.869114 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec5f04-5f88-442a-bcdb-7a06971e6f62 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:43.926682 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2ba497-7825-4171-8bd0-78717e166c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:43.927265 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ba497-7825-4171-8bd0-78717e166c00 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:43.927554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ba497-7825-4171-8bd0-78717e166c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:43.927863 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ba497-7825-4171-8bd0-78717e166c00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:44.036307 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2ba497-7825-4171-8bd0-78717e166c00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:44.037810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2ba497-7825-4171-8bd0-78717e166c00 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:44.079892 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:44.177284 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 74/147] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:14:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 07 15:14:44.182579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c904cb-5de4-4239-9c7d-c6e9dc381790 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:44.295766 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c904cb-5de4-4239-9c7d-c6e9dc381790 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:44.332539 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40894c1-d0ee-43f5-b5f9-f7fe770868a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:44.333399 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40894c1-d0ee-43f5-b5f9-f7fe770868a7 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:44.333683 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40894c1-d0ee-43f5-b5f9-f7fe770868a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:44.333961 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40894c1-d0ee-43f5-b5f9-f7fe770868a7 None admin] PATH_INFO: `/v3/projects/db427256e2ec4ec084f762a615d0f187` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:44.335168 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40894c1-d0ee-43f5-b5f9-f7fe770868a7 None admin] RBAC: Authorizing `identity:get_project(project_id=db427256e2ec4ec084f762a615d0f187)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:44.336661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40894c1-d0ee-43f5-b5f9-f7fe770868a7 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:44.338378 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 74/148] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:14:44 2022] GET /identity/v3/projects/db427256e2ec4ec084f762a615d0f187 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:44.344220 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2ba497-7825-4171-8bd0-78717e166c00 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:44.467686 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2ba497-7825-4171-8bd0-78717e166c00 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:44.497707 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:44.509600 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aee719-48d3-4eb9-bb4d-b5688815ece6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:44.526274 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aee719-48d3-4eb9-bb4d-b5688815ece6 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:44.526274 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aee719-48d3-4eb9-bb4d-b5688815ece6 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:44.526274 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aee719-48d3-4eb9-bb4d-b5688815ece6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:44.526274 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aee719-48d3-4eb9-bb4d-b5688815ece6 None admin] RBAC: Adding query filter params (role.id=0e7793fbf3064037984da651879e3a94, scope.project.id=db427256e2ec4ec084f762a615d0f187, user.id=30014035e7d641d5aecffdbe884d3d52) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:44.526274 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aee719-48d3-4eb9-bb4d-b5688815ece6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:44.526274 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aee719-48d3-4eb9-bb4d-b5688815ece6 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:44.526772 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 75/149] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:14:44 2022] GET /identity/v3/role_assignments?user.id=30014035e7d641d5aecffdbe884d3d52&scope.project.id=db427256e2ec4ec084f762a615d0f187&role.id=0e7793fbf3064037984da651879e3a94 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:45.204898 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8fa0b-8c81-489a-92fd-615fd23e94a1 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:45.205288 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8fa0b-8c81-489a-92fd-615fd23e94a1 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:45.205493 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8fa0b-8c81-489a-92fd-615fd23e94a1 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:45.207062 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 75/150] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:14:45.217346 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5dc2f6-3f92-48b4-809b-1f40ad6e09b8 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:45.217612 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5dc2f6-3f92-48b4-809b-1f40ad6e09b8 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:45.217850 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5dc2f6-3f92-48b4-809b-1f40ad6e09b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:45.242052 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-8a5dc2f6-3f92-48b4-809b-1f40ad6e09b8 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:45.243260 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-8a5dc2f6-3f92-48b4-809b-1f40ad6e09b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:45.319869 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 76/151] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:14:45.358253 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d8fa0b-8c81-489a-92fd-615fd23e94a1 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:45.399611 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:45.443161 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf7578-048c-41c5-993a-deacc6b7615c None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:45.443875 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf7578-048c-41c5-993a-deacc6b7615c None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:45.444349 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf7578-048c-41c5-993a-deacc6b7615c None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:45.445994 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 77/152] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:14:45.453157 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2efced9-6eae-4fa7-9f90-0198b934b974 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:45.453758 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2efced9-6eae-4fa7-9f90-0198b934b974 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:45.454226 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2efced9-6eae-4fa7-9f90-0198b934b974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:45.477243 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-b2efced9-6eae-4fa7-9f90-0198b934b974 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:45.479295 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-b2efced9-6eae-4fa7-9f90-0198b934b974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:45.496751 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d8fa0b-8c81-489a-92fd-615fd23e94a1 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:45.537074 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba1f334-52e8-4e95-8e70-f0d96472ef67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:45.538030 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1f334-52e8-4e95-8e70-f0d96472ef67 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:45.538320 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1f334-52e8-4e95-8e70-f0d96472ef67 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:45.538595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1f334-52e8-4e95-8e70-f0d96472ef67 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:45.551152 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba1f334-52e8-4e95-8e70-f0d96472ef67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:14:45.559471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-8ba1f334-52e8-4e95-8e70-f0d96472ef67 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 07 15:14:45.559471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:14:45.559471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:14:45.559471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:14:45.559471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:14:45.559471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:14:45.559471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:14:45.559471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:14:45.559471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:14:45.559471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:14:45.559471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:14:45.559471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:14:45.559471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:14:45.559471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:14:45.559471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:14:45.559471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:14:45.560428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:14:45.560428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:14:45.560428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:14:45.560428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:14:45.560428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:14:45.560428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:14:45.560428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:14:45.560428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:14:45.560428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:14:45.560428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:14:45.560428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:14:45.560428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:14:45.560428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:14:45.560428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:14:45.560428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:14:45.560428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:14:45.561192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:14:45.561192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:14:45.561192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:14:45.561192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:14:45.561192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:14:45.561192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:14:45.561192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:14:45.561192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:14:45.561192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:14:45.561192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:14:45.561192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 07 15:14:45.561192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:14:45.562393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 76/153] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:14:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:14:45.565604 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521799b6-6a2b-48d2-a7f3-6d90e6774f3e None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:45.565604 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521799b6-6a2b-48d2-a7f3-6d90e6774f3e None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:45.566044 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521799b6-6a2b-48d2-a7f3-6d90e6774f3e None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:45.567466 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 77/154] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:14:45.571281 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521799b6-6a2b-48d2-a7f3-6d90e6774f3e None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:45.629900 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 78/155] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 15:14:45.631214 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d6c04-c206-4c21-9b63-6f6fae18cde5 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:45.631603 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d6c04-c206-4c21-9b63-6f6fae18cde5 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:45.631858 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d6c04-c206-4c21-9b63-6f6fae18cde5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:45.662276 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-df0d6c04-c206-4c21-9b63-6f6fae18cde5 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:45.663479 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-df0d6c04-c206-4c21-9b63-6f6fae18cde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:45.693051 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:45.705713 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521799b6-6a2b-48d2-a7f3-6d90e6774f3e None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:45.744712 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:45.764905 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6adb03-0dd9-4dcf-9282-74cdd111715a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:45.765612 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6adb03-0dd9-4dcf-9282-74cdd111715a None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:45.765880 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6adb03-0dd9-4dcf-9282-74cdd111715a None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:45.766161 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6adb03-0dd9-4dcf-9282-74cdd111715a None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:45.766931 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6adb03-0dd9-4dcf-9282-74cdd111715a None admin] RBAC: Adding query filter params (name=member) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:45.767242 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6adb03-0dd9-4dcf-9282-74cdd111715a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:45.769215 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6adb03-0dd9-4dcf-9282-74cdd111715a None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:45.782231 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 79/156] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:14:45.783147 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d6c04-c206-4c21-9b63-6f6fae18cde5 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:45.783875 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 78/157] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Jul 7 15:14:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:45.790689 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6adb03-0dd9-4dcf-9282-74cdd111715a None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:45.913281 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6adb03-0dd9-4dcf-9282-74cdd111715a None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:45.930989 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d6c04-c206-4c21-9b63-6f6fae18cde5 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:45.949298 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b2c4c-23ad-43a8-8139-82ce50c084d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:45.950183 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b2c4c-23ad-43a8-8139-82ce50c084d3 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:45.950470 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b2c4c-23ad-43a8-8139-82ce50c084d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:45.950751 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b2c4c-23ad-43a8-8139-82ce50c084d3 None admin] PATH_INFO: `/v3/users/c4236ad3d5d949cf9d36da8484c8bf39` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:45.952115 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b2c4c-23ad-43a8-8139-82ce50c084d3 None admin] RBAC: Authorizing `identity:get_user(user_id=c4236ad3d5d949cf9d36da8484c8bf39)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:45.953699 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b2c4c-23ad-43a8-8139-82ce50c084d3 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:45.956258 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 79/158] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:14:45 2022] GET /identity/v3/users/c4236ad3d5d949cf9d36da8484c8bf39 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:14:45.956876 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b2c4c-23ad-43a8-8139-82ce50c084d3 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:46.000905 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624147c8-bcf4-46e9-852d-bda2f7c196ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:46.001638 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624147c8-bcf4-46e9-852d-bda2f7c196ac None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:46.001966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624147c8-bcf4-46e9-852d-bda2f7c196ac None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:46.002246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624147c8-bcf4-46e9-852d-bda2f7c196ac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:46.014182 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:46.024925 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624147c8-bcf4-46e9-852d-bda2f7c196ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:14:46.041071 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-624147c8-bcf4-46e9-852d-bda2f7c196ac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 07 15:14:46.041071 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:14:46.041071 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:14:46.041071 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:14:46.041071 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:14:46.041071 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:14:46.041071 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:14:46.041071 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:14:46.041071 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:14:46.041071 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:14:46.041071 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:14:46.041071 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:14:46.041071 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:14:46.041071 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:14:46.041071 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:14:46.041071 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:14:46.042111 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:14:46.042111 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:14:46.042111 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:14:46.042111 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:14:46.042111 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:14:46.042111 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:14:46.042111 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:14:46.042111 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:14:46.042111 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:14:46.042111 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:14:46.042111 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:14:46.042111 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:14:46.042111 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:14:46.042111 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:14:46.042111 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:14:46.042111 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:14:46.042896 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:14:46.042896 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:14:46.042896 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:14:46.042896 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:14:46.042896 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:14:46.042896 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:14:46.042896 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:14:46.042896 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:14:46.042896 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:14:46.042896 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:14:46.042896 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 07 15:14:46.042896 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:14:46.042896 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 80/159] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:14:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:14:46.043653 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624147c8-bcf4-46e9-852d-bda2f7c196ac None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:46.110162 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:46.144221 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b2c4c-23ad-43a8-8139-82ce50c084d3 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:46.195869 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624147c8-bcf4-46e9-852d-bda2f7c196ac None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:46.209599 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034fa0ae-70d9-4575-800c-ad8c5fd2c13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:46.210699 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034fa0ae-70d9-4575-800c-ad8c5fd2c13f None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:46.211190 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034fa0ae-70d9-4575-800c-ad8c5fd2c13f None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:46.211691 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034fa0ae-70d9-4575-800c-ad8c5fd2c13f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:46.223596 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034fa0ae-70d9-4575-800c-ad8c5fd2c13f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:14:46.232856 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-034fa0ae-70d9-4575-800c-ad8c5fd2c13f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 15:14:46.232856 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:14:46.232856 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:14:46.232856 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:14:46.232856 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:14:46.232856 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:14:46.232856 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:14:46.232856 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:14:46.232856 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:14:46.232856 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:14:46.232856 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:14:46.232856 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:14:46.232856 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:14:46.232856 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:14:46.232856 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:14:46.232856 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:14:46.233775 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:14:46.233775 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:14:46.233775 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:14:46.233775 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:14:46.233775 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:14:46.233775 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:14:46.233775 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:14:46.233775 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:14:46.233775 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:14:46.233775 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:14:46.233775 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:14:46.233775 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:14:46.233775 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:14:46.233775 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:14:46.233775 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:14:46.233775 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:14:46.234557 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:14:46.234557 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:14:46.234557 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:14:46.234557 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:14:46.234557 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:14:46.234557 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:14:46.234557 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:14:46.234557 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:14:46.234557 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:14:46.234557 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:14:46.234557 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 15:14:46.234557 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:14:46.237268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 80/160] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Jul 7 15:14:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:14:46.238265 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034fa0ae-70d9-4575-800c-ad8c5fd2c13f None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:46.282214 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671315e-7e82-45f5-9393-790bb4eb7e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:46.282783 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671315e-7e82-45f5-9393-790bb4eb7e9e None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:46.283109 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671315e-7e82-45f5-9393-790bb4eb7e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:46.283360 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671315e-7e82-45f5-9393-790bb4eb7e9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:46.383931 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:46.406052 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671315e-7e82-45f5-9393-790bb4eb7e9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:46.407526 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671315e-7e82-45f5-9393-790bb4eb7e9e None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:46.418466 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034fa0ae-70d9-4575-800c-ad8c5fd2c13f None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:46.489200 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733bcb1-d367-43e2-b526-4eed448e12fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:46.489888 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733bcb1-d367-43e2-b526-4eed448e12fb None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:46.490170 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733bcb1-d367-43e2-b526-4eed448e12fb None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:46.490456 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733bcb1-d367-43e2-b526-4eed448e12fb None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:46.491230 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9733bcb1-d367-43e2-b526-4eed448e12fb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:46.491541 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9733bcb1-d367-43e2-b526-4eed448e12fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:46.492957 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9733bcb1-d367-43e2-b526-4eed448e12fb None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:46.519281 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 81/161] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Jul 7 15:14:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:46.519486 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:46.520367 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b3d45-f16a-4114-9f70-fb5c21cf400e None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:46.520367 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b3d45-f16a-4114-9f70-fb5c21cf400e None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:46.520367 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b3d45-f16a-4114-9f70-fb5c21cf400e None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:46.520367 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 82/162] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:14:46.520367 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b3d45-f16a-4114-9f70-fb5c21cf400e None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:46.539151 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 81/163] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:14:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 15:14:46.542978 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671315e-7e82-45f5-9393-790bb4eb7e9e None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:46.705425 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671315e-7e82-45f5-9393-790bb4eb7e9e None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:46.747592 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b3d45-f16a-4114-9f70-fb5c21cf400e None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:46.772621 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b71b60-272f-48dc-8f33-7817f94fb0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:46.773492 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b71b60-272f-48dc-8f33-7817f94fb0ac None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:46.773774 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b71b60-272f-48dc-8f33-7817f94fb0ac None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:46.774049 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b71b60-272f-48dc-8f33-7817f94fb0ac None admin] PATH_INFO: `/v3/users/95a4c061562d43ad91c3d9188ae2c869` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:46.776056 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b71b60-272f-48dc-8f33-7817f94fb0ac None admin] RBAC: Authorizing `identity:get_user(user_id=95a4c061562d43ad91c3d9188ae2c869)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:46.777513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b71b60-272f-48dc-8f33-7817f94fb0ac None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:46.779343 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 82/164] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:14:46 2022] GET /identity/v3/users/95a4c061562d43ad91c3d9188ae2c869 => generated 289 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:46.780288 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b075d-6a0e-4106-a2a5-967e19d2b0d2 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:46.780573 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b075d-6a0e-4106-a2a5-967e19d2b0d2 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:46.780845 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b075d-6a0e-4106-a2a5-967e19d2b0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:46.802258 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:46.808775 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-f96b075d-6a0e-4106-a2a5-967e19d2b0d2 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:46.809973 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-f96b075d-6a0e-4106-a2a5-967e19d2b0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:46.813733 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea1da90-5581-4339-bb44-1ee6d8f85ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:46.824708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1da90-5581-4339-bb44-1ee6d8f85ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:46.824708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1da90-5581-4339-bb44-1ee6d8f85ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:46.824708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1da90-5581-4339-bb44-1ee6d8f85ca9 None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:46.824708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea1da90-5581-4339-bb44-1ee6d8f85ca9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:46.824708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea1da90-5581-4339-bb44-1ee6d8f85ca9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:46.824708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea1da90-5581-4339-bb44-1ee6d8f85ca9 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:46.835779 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 83/165] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:14:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:46.836430 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea1da90-5581-4339-bb44-1ee6d8f85ca9 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:46.853934 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:46.935712 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 83/166] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:14:46.936364 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b075d-6a0e-4106-a2a5-967e19d2b0d2 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:47.031837 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea1da90-5581-4339-bb44-1ee6d8f85ca9 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:47.100624 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888f381b-b7b7-4549-9035-77cd688dd4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:47.101498 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f381b-b7b7-4549-9035-77cd688dd4d9 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:47.101780 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f381b-b7b7-4549-9035-77cd688dd4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:47.102052 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f381b-b7b7-4549-9035-77cd688dd4d9 None admin] PATH_INFO: `/v3/projects/db427256e2ec4ec084f762a615d0f187` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:47.104352 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888f381b-b7b7-4549-9035-77cd688dd4d9 None admin] RBAC: Authorizing `identity:get_project(project_id=db427256e2ec4ec084f762a615d0f187)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:47.106005 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888f381b-b7b7-4549-9035-77cd688dd4d9 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:47.107969 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 84/167] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:14:46 2022] GET /identity/v3/projects/db427256e2ec4ec084f762a615d0f187 => generated 298 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:47.109941 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888f381b-b7b7-4549-9035-77cd688dd4d9 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:47.119434 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b075d-6a0e-4106-a2a5-967e19d2b0d2 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:47.154828 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4414e87c-36e4-48ee-9efd-5d68d5c188da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:47.155747 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4414e87c-36e4-48ee-9efd-5d68d5c188da None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:47.156034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4414e87c-36e4-48ee-9efd-5d68d5c188da None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:47.156308 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4414e87c-36e4-48ee-9efd-5d68d5c188da None admin] PATH_INFO: `/v3/projects/db427256e2ec4ec084f762a615d0f187` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:47.157558 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4414e87c-36e4-48ee-9efd-5d68d5c188da None admin] RBAC: Authorizing `identity:get_project(project_id=db427256e2ec4ec084f762a615d0f187)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:47.159596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4414e87c-36e4-48ee-9efd-5d68d5c188da None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:47.161339 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 84/168] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:14:46 2022] GET /identity/v3/projects/db427256e2ec4ec084f762a615d0f187 => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:47.166062 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4414e87c-36e4-48ee-9efd-5d68d5c188da None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:47.167756 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:47.211907 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:47.261556 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888f381b-b7b7-4549-9035-77cd688dd4d9 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:47.330781 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd147f6-8f17-42de-b76c-c238ec68805f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:47.331688 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd147f6-8f17-42de-b76c-c238ec68805f None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:47.331970 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd147f6-8f17-42de-b76c-c238ec68805f None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:47.332247 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd147f6-8f17-42de-b76c-c238ec68805f None admin] PATH_INFO: `/v3/users/c4236ad3d5d949cf9d36da8484c8bf39` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:47.337154 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd147f6-8f17-42de-b76c-c238ec68805f None admin] RBAC: Authorizing `identity:get_user(user_id=c4236ad3d5d949cf9d36da8484c8bf39)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:47.338598 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd147f6-8f17-42de-b76c-c238ec68805f None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:47.343052 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 85/169] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:14:47 2022] GET /identity/v3/users/c4236ad3d5d949cf9d36da8484c8bf39 => generated 275 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:47.344677 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd147f6-8f17-42de-b76c-c238ec68805f None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:47.363376 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4414e87c-36e4-48ee-9efd-5d68d5c188da None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:47.399463 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd5aee-2a54-4c86-ac18-a7fa92555a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:47.400353 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd5aee-2a54-4c86-ac18-a7fa92555a93 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:47.400636 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd5aee-2a54-4c86-ac18-a7fa92555a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:47.400909 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd5aee-2a54-4c86-ac18-a7fa92555a93 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:47.411969 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bd5aee-2a54-4c86-ac18-a7fa92555a93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:14:47.419605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-39bd5aee-2a54-4c86-ac18-a7fa92555a93 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 07 15:14:47.419605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:14:47.419605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:14:47.419605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:14:47.419605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:14:47.419605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:14:47.419605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:14:47.419605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:14:47.419605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:14:47.419605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:14:47.419605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:14:47.419605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:14:47.419605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:14:47.419605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:14:47.419605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:14:47.419605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:14:47.420525 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:14:47.420525 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:14:47.420525 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:14:47.420525 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:14:47.420525 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:14:47.420525 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:14:47.420525 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:14:47.420525 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:14:47.420525 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:14:47.420525 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:14:47.420525 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:14:47.420525 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:14:47.420525 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:14:47.420525 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:14:47.420525 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:14:47.420525 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:14:47.421335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:14:47.421335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:14:47.421335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:14:47.421335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:14:47.421335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:14:47.421335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:14:47.421335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:14:47.421335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:14:47.421335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:14:47.421335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:14:47.421335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 07 15:14:47.421335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:14:47.421335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 85/170] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:14:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:14:47.422002 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd5aee-2a54-4c86-ac18-a7fa92555a93 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:47.461964 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd147f6-8f17-42de-b76c-c238ec68805f None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:47.488114 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:47.500110 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72e6ee-4a84-47a7-931e-712ce780a38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:47.501045 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72e6ee-4a84-47a7-931e-712ce780a38c None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:47.502495 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72e6ee-4a84-47a7-931e-712ce780a38c None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:47.502803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72e6ee-4a84-47a7-931e-712ce780a38c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:47.503882 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d72e6ee-4a84-47a7-931e-712ce780a38c None admin] RBAC: Adding query filter params (role.id=d43d0d189e254500875f5f86e6abd178, scope.project.id=db427256e2ec4ec084f762a615d0f187, user.id=c4236ad3d5d949cf9d36da8484c8bf39) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:47.504398 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d72e6ee-4a84-47a7-931e-712ce780a38c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:47.506041 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d72e6ee-4a84-47a7-931e-712ce780a38c None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:47.512482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 86/171] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:14:47 2022] GET /identity/v3/role_assignments?user.id=c4236ad3d5d949cf9d36da8484c8bf39&scope.project.id=db427256e2ec4ec084f762a615d0f187&role.id=d43d0d189e254500875f5f86e6abd178 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:47.513526 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72e6ee-4a84-47a7-931e-712ce780a38c None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:47.523337 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:47.542353 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd5aee-2a54-4c86-ac18-a7fa92555a93 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:47.613915 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd620a2-87a1-4582-ac8e-4a25a682db6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:47.614772 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd620a2-87a1-4582-ac8e-4a25a682db6b None admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:47.615401 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd620a2-87a1-4582-ac8e-4a25a682db6b None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:47.615401 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd620a2-87a1-4582-ac8e-4a25a682db6b None admin] PATH_INFO: `/v3/projects/db427256e2ec4ec084f762a615d0f187/users/95a4c061562d43ad91c3d9188ae2c869/roles/4fe04bd6b66542379945fab86d689ae6` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:47.620647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd620a2-87a1-4582-ac8e-4a25a682db6b None admin] RBAC: Authorizing `identity:create_grant(project_id=db427256e2ec4ec084f762a615d0f187, user_id=95a4c061562d43ad91c3d9188ae2c869, role_id=4fe04bd6b66542379945fab86d689ae6)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:47.622160 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd620a2-87a1-4582-ac8e-4a25a682db6b None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:47.628706 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72e6ee-4a84-47a7-931e-712ce780a38c None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:47.667953 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df3aee-54d0-46be-940c-df2d1c7e5bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:47.669005 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df3aee-54d0-46be-940c-df2d1c7e5bc1 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:47.669491 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df3aee-54d0-46be-940c-df2d1c7e5bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:47.669964 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df3aee-54d0-46be-940c-df2d1c7e5bc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:47.703195 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 86/172] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Jul 7 15:14:47 2022] PUT /identity/v3/projects/db427256e2ec4ec084f762a615d0f187/users/95a4c061562d43ad91c3d9188ae2c869/roles/4fe04bd6b66542379945fab86d689ae6 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:14:47.704699 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a59eee-3051-4b24-9b82-80bdcf4716e4 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:47.705014 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a59eee-3051-4b24-9b82-80bdcf4716e4 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:47.705345 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a59eee-3051-4b24-9b82-80bdcf4716e4 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:47.711393 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 87/173] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:14:47.714374 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a59eee-3051-4b24-9b82-80bdcf4716e4 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:47.747810 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18df3aee-54d0-46be-940c-df2d1c7e5bc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:47.749751 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18df3aee-54d0-46be-940c-df2d1c7e5bc1 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:47.763918 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:47.872729 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 87/174] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:14:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 15:14:47.874391 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa047d-7779-4697-8ffb-e7940d5ea5ba None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:47.874893 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa047d-7779-4697-8ffb-e7940d5ea5ba None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:47.875383 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa047d-7779-4697-8ffb-e7940d5ea5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:47.894110 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-d6fa047d-7779-4697-8ffb-e7940d5ea5ba None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:47.895793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-d6fa047d-7779-4697-8ffb-e7940d5ea5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:47.901430 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a59eee-3051-4b24-9b82-80bdcf4716e4 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:47.942319 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:47.962741 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1fe8b-4427-497c-8c80-bde32145dc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:47.963685 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1fe8b-4427-497c-8c80-bde32145dc9f None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:47.964116 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1fe8b-4427-497c-8c80-bde32145dc9f None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:47.964534 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1fe8b-4427-497c-8c80-bde32145dc9f None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:47.965450 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc1fe8b-4427-497c-8c80-bde32145dc9f None admin] RBAC: Adding query filter params (name=member) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:47.965933 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc1fe8b-4427-497c-8c80-bde32145dc9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:47.967495 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc1fe8b-4427-497c-8c80-bde32145dc9f None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:47.980628 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 88/175] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:14:47.981523 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa047d-7779-4697-8ffb-e7940d5ea5ba None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:47.984117 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 88/176] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Jul 7 15:14:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:47.990580 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1fe8b-4427-497c-8c80-bde32145dc9f None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:48.047937 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:48.176418 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa047d-7779-4697-8ffb-e7940d5ea5ba None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:48.187241 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1fe8b-4427-497c-8c80-bde32145dc9f None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:48.221118 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e00e25c-7e13-44a3-8626-63691f5d470d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:48.222264 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00e25c-7e13-44a3-8626-63691f5d470d None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:48.222859 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00e25c-7e13-44a3-8626-63691f5d470d None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:48.223091 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00e25c-7e13-44a3-8626-63691f5d470d None admin] PATH_INFO: `/v3/projects/db427256e2ec4ec084f762a615d0f187` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:48.224315 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e00e25c-7e13-44a3-8626-63691f5d470d None admin] RBAC: Authorizing `identity:get_project(project_id=db427256e2ec4ec084f762a615d0f187)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:48.225774 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e00e25c-7e13-44a3-8626-63691f5d470d None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:48.227077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9956d526-3f2f-404b-80f9-f34e4c99b1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:48.234107 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 89/177] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:14:47 2022] GET /identity/v3/projects/db427256e2ec4ec084f762a615d0f187 => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:48.234107 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e00e25c-7e13-44a3-8626-63691f5d470d None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:48.234320 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956d526-3f2f-404b-80f9-f34e4c99b1ce None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:48.234320 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956d526-3f2f-404b-80f9-f34e4c99b1ce None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:48.234320 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956d526-3f2f-404b-80f9-f34e4c99b1ce None admin] PATH_INFO: `/v3/users/c4236ad3d5d949cf9d36da8484c8bf39` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:48.234818 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9956d526-3f2f-404b-80f9-f34e4c99b1ce None admin] RBAC: Authorizing `identity:get_user(user_id=c4236ad3d5d949cf9d36da8484c8bf39)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:48.236353 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9956d526-3f2f-404b-80f9-f34e4c99b1ce None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:48.238156 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 89/178] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:14:47 2022] GET /identity/v3/users/c4236ad3d5d949cf9d36da8484c8bf39 => generated 275 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:48.243765 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345a237-d671-47fd-aea8-d15e76a250ae None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:48.244059 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345a237-d671-47fd-aea8-d15e76a250ae None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:48.244344 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345a237-d671-47fd-aea8-d15e76a250ae None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:48.245788 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 90/179] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:14:48.246736 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6026f-e380-408a-8bec-64af4d57c45f None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:48.247044 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6026f-e380-408a-8bec-64af4d57c45f None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:48.247296 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6026f-e380-408a-8bec-64af4d57c45f None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:48.253138 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 91/180] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:14:48.256417 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6026f-e380-408a-8bec-64af4d57c45f None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:48.320173 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:48.364723 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e00e25c-7e13-44a3-8626-63691f5d470d None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:48.367181 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:48.374456 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6026f-e380-408a-8bec-64af4d57c45f None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:48.406664 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2e75c-f541-43e0-8224-ce4dfa0eccc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:48.407455 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2e75c-f541-43e0-8224-ce4dfa0eccc7 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:48.407768 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2e75c-f541-43e0-8224-ce4dfa0eccc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:48.408087 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2e75c-f541-43e0-8224-ce4dfa0eccc7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:48.415685 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb1d37-151f-4831-87bd-89492d67e6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:48.416836 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb1d37-151f-4831-87bd-89492d67e6b2 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:48.417288 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb1d37-151f-4831-87bd-89492d67e6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:48.417720 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb1d37-151f-4831-87bd-89492d67e6b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:48.418992 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cb1d37-151f-4831-87bd-89492d67e6b2 None admin] RBAC: Adding query filter params (role.id=c82203e4734346719cafb16230f45942, scope.project.id=db427256e2ec4ec084f762a615d0f187, user.id=c4236ad3d5d949cf9d36da8484c8bf39) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:48.419108 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e2e75c-f541-43e0-8224-ce4dfa0eccc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:14:48.419544 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cb1d37-151f-4831-87bd-89492d67e6b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:48.421263 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cb1d37-151f-4831-87bd-89492d67e6b2 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:48.427009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-57e2e75c-f541-43e0-8224-ce4dfa0eccc7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:14:48.427009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:14:48.427009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:14:48.427009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:14:48.427009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:14:48.427009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:14:48.427009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:14:48.427009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:14:48.427009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:14:48.427009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:14:48.427009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:14:48.427009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:14:48.427009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:14:48.427009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:14:48.427009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:14:48.427009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:14:48.428112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:14:48.428112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:14:48.428112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:14:48.428112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:14:48.428112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:14:48.428112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:14:48.428112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:14:48.428112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:14:48.428112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:14:48.428112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:14:48.428112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:14:48.428112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:14:48.428112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:14:48.428112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:14:48.428112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:14:48.428112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:14:48.429029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:14:48.429029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:14:48.429029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:14:48.429029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:14:48.429029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:14:48.429029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:14:48.429029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:14:48.429029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:14:48.429029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:14:48.429029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:14:48.429029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:14:48.429029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:14:48.429029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 90/181] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Jul 7 15:14:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:14:48.429029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2e75c-f541-43e0-8224-ce4dfa0eccc7 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:48.430835 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 92/182] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:14:48 2022] GET /identity/v3/role_assignments?user.id=c4236ad3d5d949cf9d36da8484c8bf39&scope.project.id=db427256e2ec4ec084f762a615d0f187&role.id=c82203e4734346719cafb16230f45942 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:48.432450 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774fd06-b8c5-40a8-b561-b4ba0ea11a2b None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:48.432867 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774fd06-b8c5-40a8-b561-b4ba0ea11a2b None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:48.433280 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774fd06-b8c5-40a8-b561-b4ba0ea11a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:48.461943 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-7774fd06-b8c5-40a8-b561-b4ba0ea11a2b None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:48.463515 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-7774fd06-b8c5-40a8-b561-b4ba0ea11a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:48.563360 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 93/183] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:14:48.565374 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2e75c-f541-43e0-8224-ce4dfa0eccc7 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:48.566566 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3300131-45c4-4001-aa7c-af1d9a51b775 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:48.567005 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3300131-45c4-4001-aa7c-af1d9a51b775 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:48.567533 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3300131-45c4-4001-aa7c-af1d9a51b775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:48.586221 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-f3300131-45c4-4001-aa7c-af1d9a51b775 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:48.587773 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-f3300131-45c4-4001-aa7c-af1d9a51b775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:48.614986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a55457-2f88-4d45-a51f-77c00ed270d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:48.615674 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a55457-2f88-4d45-a51f-77c00ed270d7 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:48.615957 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a55457-2f88-4d45-a51f-77c00ed270d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:48.616396 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a55457-2f88-4d45-a51f-77c00ed270d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:48.641188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:48.655610 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:48.675169 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 94/184] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:14:48.680662 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3300131-45c4-4001-aa7c-af1d9a51b775 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:48.710169 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a55457-2f88-4d45-a51f-77c00ed270d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:48.711678 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a55457-2f88-4d45-a51f-77c00ed270d7 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:48.805872 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3300131-45c4-4001-aa7c-af1d9a51b775 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:48.829219 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 91/185] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:14:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 15:14:48.829899 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a55457-2f88-4d45-a51f-77c00ed270d7 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:48.852293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096a8a4-e34c-4e35-8462-cc3ab41600fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:48.853240 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096a8a4-e34c-4e35-8462-cc3ab41600fb None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:48.853527 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096a8a4-e34c-4e35-8462-cc3ab41600fb None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:48.853799 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096a8a4-e34c-4e35-8462-cc3ab41600fb None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:48.854816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0096a8a4-e34c-4e35-8462-cc3ab41600fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:48.855523 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0096a8a4-e34c-4e35-8462-cc3ab41600fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:48.857466 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0096a8a4-e34c-4e35-8462-cc3ab41600fb None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:48.868459 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 95/186] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:14:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:48.869874 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096a8a4-e34c-4e35-8462-cc3ab41600fb None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:48.897742 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:48.953544 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:48.980706 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a55457-2f88-4d45-a51f-77c00ed270d7 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:49.018325 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52aee4f-dade-4937-b5c6-8cd032264416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:49.019115 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52aee4f-dade-4937-b5c6-8cd032264416 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:49.019602 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52aee4f-dade-4937-b5c6-8cd032264416 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:49.019710 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52aee4f-dade-4937-b5c6-8cd032264416 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:49.020869 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52aee4f-dade-4937-b5c6-8cd032264416 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:49.022322 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52aee4f-dade-4937-b5c6-8cd032264416 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:49.022422 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096a8a4-e34c-4e35-8462-cc3ab41600fb None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:49.023996 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 92/187] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:14:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:49.024756 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52aee4f-dade-4937-b5c6-8cd032264416 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:49.063256 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6aa998-b611-4559-9445-a71914079ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:49.064094 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6aa998-b611-4559-9445-a71914079ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:49.064964 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6aa998-b611-4559-9445-a71914079ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:49.065237 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6aa998-b611-4559-9445-a71914079ab8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:49.075758 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6aa998-b611-4559-9445-a71914079ab8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:14:49.084035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-1d6aa998-b611-4559-9445-a71914079ab8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 15:14:49.084035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:14:49.084035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:14:49.084035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:14:49.084035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:14:49.084035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:14:49.084035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:14:49.084035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:14:49.084035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:14:49.084035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:14:49.084035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:14:49.084035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:14:49.084035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:14:49.084035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:14:49.084035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:14:49.084035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:14:49.085178 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:14:49.085178 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:14:49.085178 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:14:49.085178 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:14:49.085178 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:14:49.085178 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:14:49.085178 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:14:49.085178 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:14:49.085178 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:14:49.085178 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:14:49.085178 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:14:49.085178 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:14:49.085178 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:14:49.085178 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:14:49.085178 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:14:49.085178 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:14:49.086043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:14:49.086043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:14:49.086043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:14:49.086043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:14:49.086043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:14:49.086043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:14:49.086043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:14:49.086043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:14:49.086043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:14:49.086043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:14:49.086043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 15:14:49.086043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:14:49.086043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 96/188] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Jul 7 15:14:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:14:49.086714 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6aa998-b611-4559-9445-a71914079ab8 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:49.141329 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52aee4f-dade-4937-b5c6-8cd032264416 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:49.174392 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:49.179405 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c28bc-002c-44e1-99a4-46e7f1c364c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:49.180310 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c28bc-002c-44e1-99a4-46e7f1c364c8 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:49.180596 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c28bc-002c-44e1-99a4-46e7f1c364c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:49.183019 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c28bc-002c-44e1-99a4-46e7f1c364c8 None admin] PATH_INFO: `/v3/projects/cfaa23d7de8a4cbf96d2831cea78f1b1` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:49.183019 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c28bc-002c-44e1-99a4-46e7f1c364c8 None admin] RBAC: Authorizing `identity:get_project(project_id=cfaa23d7de8a4cbf96d2831cea78f1b1)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:49.183499 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c28bc-002c-44e1-99a4-46e7f1c364c8 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:49.190609 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 93/189] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:14:49 2022] GET /identity/v3/projects/cfaa23d7de8a4cbf96d2831cea78f1b1 => generated 312 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:49.190609 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c28bc-002c-44e1-99a4-46e7f1c364c8 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:49.197610 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:49.200541 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6aa998-b611-4559-9445-a71914079ab8 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:49.236093 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b69a1-d170-4578-8cbd-fed494a91f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:49.237449 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b69a1-d170-4578-8cbd-fed494a91f84 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:49.238136 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b69a1-d170-4578-8cbd-fed494a91f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:49.239206 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b69a1-d170-4578-8cbd-fed494a91f84 None admin] PATH_INFO: `/v3/users/8ccc20f4d89b4c7196bc132e6e504f12` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:49.254389 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826b69a1-d170-4578-8cbd-fed494a91f84 None admin] RBAC: Authorizing `identity:get_user(user_id=8ccc20f4d89b4c7196bc132e6e504f12)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:49.255867 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826b69a1-d170-4578-8cbd-fed494a91f84 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:49.257666 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 97/190] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:14:49 2022] GET /identity/v3/users/8ccc20f4d89b4c7196bc132e6e504f12 => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:49.258340 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b69a1-d170-4578-8cbd-fed494a91f84 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:49.298277 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c28bc-002c-44e1-99a4-46e7f1c364c8 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:49.332197 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ed281a-2703-4846-871c-344338f370f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:49.332860 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed281a-2703-4846-871c-344338f370f4 None admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:49.333138 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed281a-2703-4846-871c-344338f370f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:49.333409 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed281a-2703-4846-871c-344338f370f4 None admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:49.334115 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ed281a-2703-4846-871c-344338f370f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:49.335604 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ed281a-2703-4846-871c-344338f370f4 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:49.371288 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b69a1-d170-4578-8cbd-fed494a91f84 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:49.405378 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21940e-12b7-4b72-aa01-7dcf0a191db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:49.406032 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21940e-12b7-4b72-aa01-7dcf0a191db1 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:49.406315 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21940e-12b7-4b72-aa01-7dcf0a191db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:49.406598 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21940e-12b7-4b72-aa01-7dcf0a191db1 None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:49.407342 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21940e-12b7-4b72-aa01-7dcf0a191db1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:49.407683 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21940e-12b7-4b72-aa01-7dcf0a191db1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:49.409087 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21940e-12b7-4b72-aa01-7dcf0a191db1 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:49.417199 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 98/191] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:14:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:49.417988 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21940e-12b7-4b72-aa01-7dcf0a191db1 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:49.439091 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 94/192] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Jul 7 15:14:49 2022] POST /identity/v3/users => generated 297 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:14:49.440469 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ed281a-2703-4846-871c-344338f370f4 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:49.440598 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:49.510307 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:49.559534 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ed281a-2703-4846-871c-344338f370f4 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:49.571131 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21940e-12b7-4b72-aa01-7dcf0a191db1 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:49.613224 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49967d45-b692-4d55-a1dc-58fd628571e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:49.613884 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49967d45-b692-4d55-a1dc-58fd628571e0 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:49.614167 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49967d45-b692-4d55-a1dc-58fd628571e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:49.614447 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49967d45-b692-4d55-a1dc-58fd628571e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:49.624245 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd3cff-d9f7-4ed4-89d3-44d8295d1566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:49.625264 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd3cff-d9f7-4ed4-89d3-44d8295d1566 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:49.626443 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd3cff-d9f7-4ed4-89d3-44d8295d1566 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:49.627604 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd3cff-d9f7-4ed4-89d3-44d8295d1566 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:49.629082 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fd3cff-d9f7-4ed4-89d3-44d8295d1566 None admin] RBAC: Adding query filter params (role.id=d43d0d189e254500875f5f86e6abd178, scope.project.id=cfaa23d7de8a4cbf96d2831cea78f1b1, user.id=c4236ad3d5d949cf9d36da8484c8bf39) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:49.629209 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fd3cff-d9f7-4ed4-89d3-44d8295d1566 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:49.630649 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fd3cff-d9f7-4ed4-89d3-44d8295d1566 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:49.637937 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 99/193] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:14:49 2022] GET /identity/v3/role_assignments?user.id=c4236ad3d5d949cf9d36da8484c8bf39&scope.project.id=cfaa23d7de8a4cbf96d2831cea78f1b1&role.id=d43d0d189e254500875f5f86e6abd178 => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:49.638754 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd3cff-d9f7-4ed4-89d3-44d8295d1566 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:49.700321 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49967d45-b692-4d55-a1dc-58fd628571e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:49.702239 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49967d45-b692-4d55-a1dc-58fd628571e0 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:49.771083 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:49.774740 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:49.795167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd3cff-d9f7-4ed4-89d3-44d8295d1566 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:49.841372 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 95/194] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:14:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 15:14:49.848173 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49967d45-b692-4d55-a1dc-58fd628571e0 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:49.859475 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf87c3-c4ef-484c-9660-133464fca188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:49.860472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf87c3-c4ef-484c-9660-133464fca188 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:49.860758 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf87c3-c4ef-484c-9660-133464fca188 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:49.863050 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf87c3-c4ef-484c-9660-133464fca188 None admin] PATH_INFO: `/v3/users/8ccc20f4d89b4c7196bc132e6e504f12` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:49.864410 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bf87c3-c4ef-484c-9660-133464fca188 None admin] RBAC: Authorizing `identity:get_user(user_id=8ccc20f4d89b4c7196bc132e6e504f12)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:49.869490 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bf87c3-c4ef-484c-9660-133464fca188 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:49.869490 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 100/195] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:14:49 2022] GET /identity/v3/users/8ccc20f4d89b4c7196bc132e6e504f12 => generated 283 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:49.873718 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf87c3-c4ef-484c-9660-133464fca188 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:49.992223 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49967d45-b692-4d55-a1dc-58fd628571e0 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:50.005152 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf87c3-c4ef-484c-9660-133464fca188 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:50.039925 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97759512-6d38-4ee8-a3d6-384bd4ac4d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:50.041293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97759512-6d38-4ee8-a3d6-384bd4ac4d07 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:50.041722 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97759512-6d38-4ee8-a3d6-384bd4ac4d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:50.042143 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97759512-6d38-4ee8-a3d6-384bd4ac4d07 None admin] PATH_INFO: `/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:50.043181 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23927ca3-15e5-4ef5-a94a-7414d49c7888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:50.043822 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97759512-6d38-4ee8-a3d6-384bd4ac4d07 None admin] RBAC: Authorizing `identity:get_project(project_id=a6fde692f8db4d7eb4858942b0b6e5bf)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:50.043934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23927ca3-15e5-4ef5-a94a-7414d49c7888 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:50.044150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23927ca3-15e5-4ef5-a94a-7414d49c7888 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:50.044430 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23927ca3-15e5-4ef5-a94a-7414d49c7888 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:50.045810 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97759512-6d38-4ee8-a3d6-384bd4ac4d07 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:50.049536 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 96/196] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:14:49 2022] GET /identity/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:50.055757 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97759512-6d38-4ee8-a3d6-384bd4ac4d07 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:50.070804 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:50.089315 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:50.136650 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23927ca3-15e5-4ef5-a94a-7414d49c7888 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:50.139153 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23927ca3-15e5-4ef5-a94a-7414d49c7888 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:50.182477 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97759512-6d38-4ee8-a3d6-384bd4ac4d07 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:50.218721 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06451f3-909e-4114-a355-6729bae86ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:50.219376 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06451f3-909e-4114-a355-6729bae86ddb None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:50.219687 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06451f3-909e-4114-a355-6729bae86ddb None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:50.219966 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06451f3-909e-4114-a355-6729bae86ddb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:50.221371 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06451f3-909e-4114-a355-6729bae86ddb None admin] RBAC: Adding query filter params (role.id=0e7793fbf3064037984da651879e3a94, scope.project.id=a6fde692f8db4d7eb4858942b0b6e5bf, user.id=8ccc20f4d89b4c7196bc132e6e504f12) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:50.221684 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06451f3-909e-4114-a355-6729bae86ddb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:50.223370 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06451f3-909e-4114-a355-6729bae86ddb None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:50.230343 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 97/197] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:14:50 2022] GET /identity/v3/role_assignments?user.id=8ccc20f4d89b4c7196bc132e6e504f12&scope.project.id=a6fde692f8db4d7eb4858942b0b6e5bf&role.id=0e7793fbf3064037984da651879e3a94 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:50.235715 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f8fd4-a506-44f5-a6d0-b9c570ecf7a8 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:50.235715 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f8fd4-a506-44f5-a6d0-b9c570ecf7a8 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:50.235715 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f8fd4-a506-44f5-a6d0-b9c570ecf7a8 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:50.235715 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 98/198] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:14:50.239421 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88f976-4d34-44e2-8599-05645e579820 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:50.239421 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88f976-4d34-44e2-8599-05645e579820 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:50.239745 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88f976-4d34-44e2-8599-05645e579820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:50.272528 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-4a88f976-4d34-44e2-8599-05645e579820 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:50.273755 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-4a88f976-4d34-44e2-8599-05645e579820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:50.309177 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 101/199] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:14:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 07 15:14:50.320682 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23927ca3-15e5-4ef5-a94a-7414d49c7888 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:50.411055 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 99/200] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:14:50.423375 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:50.433394 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88f976-4d34-44e2-8599-05645e579820 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:50.452098 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:50.484990 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23927ca3-15e5-4ef5-a94a-7414d49c7888 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:50.527945 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9913a968-7ba3-440e-a5a3-6fe43efc7407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:50.529161 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9913a968-7ba3-440e-a5a3-6fe43efc7407 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:50.529644 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9913a968-7ba3-440e-a5a3-6fe43efc7407 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:50.530352 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9913a968-7ba3-440e-a5a3-6fe43efc7407 None admin] PATH_INFO: `/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:50.531821 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9913a968-7ba3-440e-a5a3-6fe43efc7407 None admin] RBAC: Authorizing `identity:get_project(project_id=a6fde692f8db4d7eb4858942b0b6e5bf)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:50.534069 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9913a968-7ba3-440e-a5a3-6fe43efc7407 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:50.536467 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 102/201] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:14:50 2022] GET /identity/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:50.546548 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9913a968-7ba3-440e-a5a3-6fe43efc7407 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:50.587231 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88f976-4d34-44e2-8599-05645e579820 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:50.633114 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a340f-e720-4eda-a2ee-b2e83c53b62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:50.634214 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a340f-e720-4eda-a2ee-b2e83c53b62e None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:50.634544 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a340f-e720-4eda-a2ee-b2e83c53b62e None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:50.634823 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a340f-e720-4eda-a2ee-b2e83c53b62e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:50.648006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599a340f-e720-4eda-a2ee-b2e83c53b62e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:14:50.656336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-599a340f-e720-4eda-a2ee-b2e83c53b62e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 07 15:14:50.656336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:14:50.656336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:14:50.656336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:14:50.656336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:14:50.656336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:14:50.656336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:14:50.656336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:14:50.656336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:14:50.656336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:14:50.656336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:14:50.656336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:14:50.656336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:14:50.656336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:14:50.656336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:14:50.656336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:14:50.657293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:14:50.657293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:14:50.657293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:14:50.657293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:14:50.657293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:14:50.657293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:14:50.657293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:14:50.657293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:14:50.657293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:14:50.657293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:14:50.657293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:14:50.657293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:14:50.657293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:14:50.657293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:14:50.657293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:14:50.657293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:14:50.658135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:14:50.658135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:14:50.658135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:14:50.658135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:14:50.658135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:14:50.658135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:14:50.658135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:14:50.658135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:14:50.658135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:14:50.658135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:14:50.658135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 07 15:14:50.658135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:14:50.658740 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 100/202] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:14:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:14:50.666052 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a340f-e720-4eda-a2ee-b2e83c53b62e None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:50.724195 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9913a968-7ba3-440e-a5a3-6fe43efc7407 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:50.755480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:50.764166 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0506d8a2-39b8-4e1b-b266-d458078b4652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:50.764966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0506d8a2-39b8-4e1b-b266-d458078b4652 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:50.765253 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0506d8a2-39b8-4e1b-b266-d458078b4652 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:50.765557 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0506d8a2-39b8-4e1b-b266-d458078b4652 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:50.766361 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0506d8a2-39b8-4e1b-b266-d458078b4652 None admin] RBAC: Adding query filter params (role.id=c82203e4734346719cafb16230f45942, scope.project.id=a6fde692f8db4d7eb4858942b0b6e5bf, user.id=8ccc20f4d89b4c7196bc132e6e504f12) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:50.766674 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0506d8a2-39b8-4e1b-b266-d458078b4652 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:50.768215 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0506d8a2-39b8-4e1b-b266-d458078b4652 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:50.775518 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 103/203] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:14:50 2022] GET /identity/v3/role_assignments?user.id=8ccc20f4d89b4c7196bc132e6e504f12&scope.project.id=a6fde692f8db4d7eb4858942b0b6e5bf&role.id=c82203e4734346719cafb16230f45942 => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:14:50.818111 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:50.842403 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a340f-e720-4eda-a2ee-b2e83c53b62e None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:50.894783 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201b948-b199-4523-93cd-41fecae938bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:50.895470 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201b948-b199-4523-93cd-41fecae938bb None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:50.895830 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201b948-b199-4523-93cd-41fecae938bb None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:50.896116 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201b948-b199-4523-93cd-41fecae938bb None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:50.897881 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4201b948-b199-4523-93cd-41fecae938bb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:50.897881 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4201b948-b199-4523-93cd-41fecae938bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:50.898625 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4201b948-b199-4523-93cd-41fecae938bb None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:50.911306 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 101/204] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Jul 7 15:14:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:50.916999 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0506d8a2-39b8-4e1b-b266-d458078b4652 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:51.037752 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0506d8a2-39b8-4e1b-b266-d458078b4652 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:51.088278 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce564a48-f762-49a2-930f-38b8d3e50efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:51.089166 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce564a48-f762-49a2-930f-38b8d3e50efc None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:51.089451 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce564a48-f762-49a2-930f-38b8d3e50efc None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:51.089724 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce564a48-f762-49a2-930f-38b8d3e50efc None admin] PATH_INFO: `/v3/users/95a4c061562d43ad91c3d9188ae2c869` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:51.090966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce564a48-f762-49a2-930f-38b8d3e50efc None admin] RBAC: Authorizing `identity:get_user(user_id=95a4c061562d43ad91c3d9188ae2c869)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:51.092690 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce564a48-f762-49a2-930f-38b8d3e50efc None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:51.094482 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 104/205] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:14:50 2022] GET /identity/v3/users/95a4c061562d43ad91c3d9188ae2c869 => generated 289 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:51.100750 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201b948-b199-4523-93cd-41fecae938bb None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:51.252465 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201b948-b199-4523-93cd-41fecae938bb None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:51.310587 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad49c4-a01a-4adf-ae14-4b9be962621d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:51.311641 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad49c4-a01a-4adf-ae14-4b9be962621d None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:51.312120 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad49c4-a01a-4adf-ae14-4b9be962621d None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:51.312588 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad49c4-a01a-4adf-ae14-4b9be962621d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:51.325708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:51.399114 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad49c4-a01a-4adf-ae14-4b9be962621d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:51.400644 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad49c4-a01a-4adf-ae14-4b9be962621d None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:51.532088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 102/206] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:14:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 07 15:14:51.537704 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce564a48-f762-49a2-930f-38b8d3e50efc None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:51.612381 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:51.678786 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce564a48-f762-49a2-930f-38b8d3e50efc None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:51.727272 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a7ec8-c4f5-4051-8194-3c17a266abff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:51.728176 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a7ec8-c4f5-4051-8194-3c17a266abff None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:51.728458 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a7ec8-c4f5-4051-8194-3c17a266abff None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:51.728736 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a7ec8-c4f5-4051-8194-3c17a266abff None admin] PATH_INFO: `/v3/projects/db427256e2ec4ec084f762a615d0f187` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:51.729955 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6a7ec8-c4f5-4051-8194-3c17a266abff None admin] RBAC: Authorizing `identity:get_project(project_id=db427256e2ec4ec084f762a615d0f187)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:51.731419 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6a7ec8-c4f5-4051-8194-3c17a266abff None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:51.733461 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 105/207] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:14:51 2022] GET /identity/v3/projects/db427256e2ec4ec084f762a615d0f187 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:51.739607 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad49c4-a01a-4adf-ae14-4b9be962621d None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:51.795970 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:51.874477 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad49c4-a01a-4adf-ae14-4b9be962621d None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:51.922826 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8485d54b-95dc-40a8-ade3-c2b6c2e39dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:51.923251 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485d54b-95dc-40a8-ade3-c2b6c2e39dce None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:51.923329 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485d54b-95dc-40a8-ade3-c2b6c2e39dce None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:51.924176 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485d54b-95dc-40a8-ade3-c2b6c2e39dce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:51.925443 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8485d54b-95dc-40a8-ade3-c2b6c2e39dce None admin] RBAC: Adding query filter params (role.id=4fe04bd6b66542379945fab86d689ae6, scope.project.id=db427256e2ec4ec084f762a615d0f187, user.id=95a4c061562d43ad91c3d9188ae2c869) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:51.925443 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8485d54b-95dc-40a8-ade3-c2b6c2e39dce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:51.926876 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8485d54b-95dc-40a8-ade3-c2b6c2e39dce None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:51.939964 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 103/208] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:14:51 2022] GET /identity/v3/role_assignments?user.id=95a4c061562d43ad91c3d9188ae2c869&scope.project.id=db427256e2ec4ec084f762a615d0f187&role.id=4fe04bd6b66542379945fab86d689ae6 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:52.313618 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7c833-f2ce-40fa-913d-fb0e5e5cfc09 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:52.313957 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7c833-f2ce-40fa-913d-fb0e5e5cfc09 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:52.314176 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7c833-f2ce-40fa-913d-fb0e5e5cfc09 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:52.315792 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 106/209] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:14:52.321127 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f1b5c-f37b-410b-a9df-515b5bc0fa29 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:52.321415 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f1b5c-f37b-410b-a9df-515b5bc0fa29 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:52.321689 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f1b5c-f37b-410b-a9df-515b5bc0fa29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:52.343780 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-924f1b5c-f37b-410b-a9df-515b5bc0fa29 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:52.345851 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-924f1b5c-f37b-410b-a9df-515b5bc0fa29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:52.388927 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed56b13-df98-4427-8efc-69e588a21cc4 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:52.389548 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed56b13-df98-4427-8efc-69e588a21cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:52.389887 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed56b13-df98-4427-8efc-69e588a21cc4 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:52.391615 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 107/210] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:14:52.396972 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77865ef1-7b97-4226-b0d6-266463d7fbd3 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:52.397259 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77865ef1-7b97-4226-b0d6-266463d7fbd3 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:52.397757 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77865ef1-7b97-4226-b0d6-266463d7fbd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:52.418330 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-77865ef1-7b97-4226-b0d6-266463d7fbd3 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:52.418544 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 104/211] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:14:52.420365 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-77865ef1-7b97-4226-b0d6-266463d7fbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:52.445198 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924f1b5c-f37b-410b-a9df-515b5bc0fa29 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:52.485539 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:52.494505 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 108/212] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 15:14:52.517694 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77865ef1-7b97-4226-b0d6-266463d7fbd3 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:52.576982 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:52.582580 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924f1b5c-f37b-410b-a9df-515b5bc0fa29 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:52.639428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b71727-2ba5-4688-a83f-03d98b4c5b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:52.641261 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b71727-2ba5-4688-a83f-03d98b4c5b60 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:52.641746 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b71727-2ba5-4688-a83f-03d98b4c5b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:52.642849 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b71727-2ba5-4688-a83f-03d98b4c5b60 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:52.643298 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77865ef1-7b97-4226-b0d6-266463d7fbd3 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:52.644419 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b71727-2ba5-4688-a83f-03d98b4c5b60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:52.647189 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b71727-2ba5-4688-a83f-03d98b4c5b60 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:52.649238 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 105/213] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:14:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:14:52.654569 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b71727-2ba5-4688-a83f-03d98b4c5b60 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:52.693009 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d218f-9f60-42bb-a04a-64ef465105d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:52.693811 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d218f-9f60-42bb-a04a-64ef465105d6 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:52.694096 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d218f-9f60-42bb-a04a-64ef465105d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:52.694545 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d218f-9f60-42bb-a04a-64ef465105d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:52.710765 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83d218f-9f60-42bb-a04a-64ef465105d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:14:52.722167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-e83d218f-9f60-42bb-a04a-64ef465105d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 07 15:14:52.722167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:14:52.722167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:14:52.722167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:14:52.722167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:14:52.722167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:14:52.722167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:14:52.722167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:14:52.722167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:14:52.722167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:14:52.722167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:14:52.722167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:14:52.722167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:14:52.722167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:14:52.722167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:14:52.722167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:14:52.723351 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:14:52.723351 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:14:52.723351 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:14:52.723351 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:14:52.723351 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:14:52.723351 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:14:52.723351 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:14:52.723351 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:14:52.723351 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:14:52.723351 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:14:52.723351 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:14:52.723351 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:14:52.723351 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:14:52.723351 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:14:52.723351 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:14:52.723351 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:14:52.724638 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:14:52.724638 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:14:52.724638 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:14:52.724638 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:14:52.724638 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:14:52.724638 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:14:52.724638 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:14:52.724638 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:14:52.724638 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:14:52.724638 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:14:52.724638 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 07 15:14:52.724638 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:14:52.724638 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 109/214] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:14:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:14:52.729332 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d218f-9f60-42bb-a04a-64ef465105d6 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:52.769080 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b71727-2ba5-4688-a83f-03d98b4c5b60 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:52.771316 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:52.808261 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de9de8-c06b-49b1-9537-2f64b8824e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:52.809334 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de9de8-c06b-49b1-9537-2f64b8824e8c None admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:52.809830 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de9de8-c06b-49b1-9537-2f64b8824e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:52.810317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de9de8-c06b-49b1-9537-2f64b8824e8c None admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:52.811272 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de9de8-c06b-49b1-9537-2f64b8824e8c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:52.812980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de9de8-c06b-49b1-9537-2f64b8824e8c None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:52.838585 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d218f-9f60-42bb-a04a-64ef465105d6 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:52.849342 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:52.872607 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d88279-e113-461e-9072-9392db48e077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:52.877195 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d88279-e113-461e-9072-9392db48e077 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:52.877195 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d88279-e113-461e-9072-9392db48e077 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:52.877195 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d88279-e113-461e-9072-9392db48e077 None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:52.877195 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d88279-e113-461e-9072-9392db48e077 None admin] RBAC: Adding query filter params (name=member) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:52.877195 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d88279-e113-461e-9072-9392db48e077 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:52.877195 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d88279-e113-461e-9072-9392db48e077 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:52.885243 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 110/215] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Jul 7 15:14:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:14:52.889869 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2cf45-e8ef-4416-9507-54992ff1f13e None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:52.889869 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2cf45-e8ef-4416-9507-54992ff1f13e None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:52.889869 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2cf45-e8ef-4416-9507-54992ff1f13e None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:52.889869 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 111/216] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:14:52.890616 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da2cf45-e8ef-4416-9507-54992ff1f13e None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:52.902619 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 106/217] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Jul 7 15:14:52 2022] POST /identity/v3/users => generated 297 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:14:52.904209 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa65f0d-70b9-486a-993c-326025c82815 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:52.904701 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa65f0d-70b9-486a-993c-326025c82815 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:52.905201 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa65f0d-70b9-486a-993c-326025c82815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:52.924763 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-2fa65f0d-70b9-486a-993c-326025c82815 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:52.926593 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-2fa65f0d-70b9-486a-993c-326025c82815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:53.032309 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 107/218] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:14:53.041289 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da2cf45-e8ef-4416-9507-54992ff1f13e None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:53.054890 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa65f0d-70b9-486a-993c-326025c82815 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:53.091427 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b1d04-1159-465e-80d3-325dbdb31b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:53.092426 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b1d04-1159-465e-80d3-325dbdb31b8a None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:53.093008 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b1d04-1159-465e-80d3-325dbdb31b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:53.093293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b1d04-1159-465e-80d3-325dbdb31b8a None admin] PATH_INFO: `/v3/users/7c8dca7248a54203bfb90013cf4fac8d` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:53.111913 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627b1d04-1159-465e-80d3-325dbdb31b8a None admin] RBAC: Authorizing `identity:get_user(user_id=7c8dca7248a54203bfb90013cf4fac8d)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:53.113738 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627b1d04-1159-465e-80d3-325dbdb31b8a None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:53.115770 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 112/219] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:14:52 2022] GET /identity/v3/users/7c8dca7248a54203bfb90013cf4fac8d => generated 297 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:14:53.121045 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b1d04-1159-465e-80d3-325dbdb31b8a None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:53.155689 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:53.171693 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:53.186800 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa65f0d-70b9-486a-993c-326025c82815 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:53.232776 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f6d0a-851c-450a-a00e-b7096cb07c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:53.233585 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f6d0a-851c-450a-a00e-b7096cb07c05 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:53.233875 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f6d0a-851c-450a-a00e-b7096cb07c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:53.234149 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f6d0a-851c-450a-a00e-b7096cb07c05 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:53.246604 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8f6d0a-851c-450a-a00e-b7096cb07c05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:14:53.249595 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b1d04-1159-465e-80d3-325dbdb31b8a None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:53.255333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-fc8f6d0a-851c-450a-a00e-b7096cb07c05 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:14:53.255333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:14:53.255333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:14:53.255333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:14:53.255333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:14:53.255333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:14:53.255333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:14:53.255333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:14:53.255333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:14:53.255333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:14:53.255333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:14:53.255333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:14:53.255333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:14:53.255333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:14:53.255333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:14:53.255333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:14:53.256317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:14:53.256317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:14:53.256317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:14:53.256317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:14:53.256317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:14:53.256317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:14:53.256317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:14:53.256317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:14:53.256317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:14:53.256317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:14:53.256317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:14:53.256317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:14:53.256317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:14:53.256317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:14:53.256317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:14:53.256317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:14:53.257532 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:14:53.257532 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:14:53.257532 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:14:53.257532 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:14:53.257532 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:14:53.257532 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:14:53.257532 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:14:53.257532 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:14:53.257532 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:14:53.257532 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:14:53.257532 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:14:53.257532 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:14:53.257532 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 108/220] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Jul 7 15:14:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:14:53.258531 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf75bf-baff-4b49-ae52-a3603be68c30 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:53.258531 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf75bf-baff-4b49-ae52-a3603be68c30 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:53.259165 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf75bf-baff-4b49-ae52-a3603be68c30 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:53.260780 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 109/221] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:14:53.262916 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf75bf-baff-4b49-ae52-a3603be68c30 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:53.293357 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc437399-5899-49e5-bb50-fa437905df2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:53.294485 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc437399-5899-49e5-bb50-fa437905df2a None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:53.295772 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc437399-5899-49e5-bb50-fa437905df2a None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:53.296254 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc437399-5899-49e5-bb50-fa437905df2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:53.367636 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc437399-5899-49e5-bb50-fa437905df2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:53.369368 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc437399-5899-49e5-bb50-fa437905df2a None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:53.380937 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf75bf-baff-4b49-ae52-a3603be68c30 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:53.401011 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:53.418615 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968a3cea-47a6-44d5-8b5e-c9e5257cd276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:53.419450 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a3cea-47a6-44d5-8b5e-c9e5257cd276 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:53.419813 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a3cea-47a6-44d5-8b5e-c9e5257cd276 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:53.420191 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a3cea-47a6-44d5-8b5e-c9e5257cd276 None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:53.421229 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968a3cea-47a6-44d5-8b5e-c9e5257cd276 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:53.421532 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968a3cea-47a6-44d5-8b5e-c9e5257cd276 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:53.423023 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968a3cea-47a6-44d5-8b5e-c9e5257cd276 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:53.432599 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 110/222] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:14:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:53.433984 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430978e3-47d5-4a28-a2fe-8c8ef961bb75 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:53.434344 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430978e3-47d5-4a28-a2fe-8c8ef961bb75 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:53.434686 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430978e3-47d5-4a28-a2fe-8c8ef961bb75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:53.458934 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-430978e3-47d5-4a28-a2fe-8c8ef961bb75 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:53.460088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-430978e3-47d5-4a28-a2fe-8c8ef961bb75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:53.463856 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:53.479341 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 113/223] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:14:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 07 15:14:53.480521 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc437399-5899-49e5-bb50-fa437905df2a None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:53.537775 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 111/224] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:14:53.543883 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430978e3-47d5-4a28-a2fe-8c8ef961bb75 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:53.612827 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc437399-5899-49e5-bb50-fa437905df2a None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:53.646695 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430978e3-47d5-4a28-a2fe-8c8ef961bb75 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:53.650398 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:53.657601 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba1a22-60a9-4a6f-ad89-91c52ba64e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:53.659623 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba1a22-60a9-4a6f-ad89-91c52ba64e5c None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:53.665288 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba1a22-60a9-4a6f-ad89-91c52ba64e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:53.665288 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba1a22-60a9-4a6f-ad89-91c52ba64e5c None admin] PATH_INFO: `/v3/users/8ccc20f4d89b4c7196bc132e6e504f12` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:53.665288 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ba1a22-60a9-4a6f-ad89-91c52ba64e5c None admin] RBAC: Authorizing `identity:get_user(user_id=8ccc20f4d89b4c7196bc132e6e504f12)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:53.665288 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ba1a22-60a9-4a6f-ad89-91c52ba64e5c None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:53.665864 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 114/225] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:14:53 2022] GET /identity/v3/users/8ccc20f4d89b4c7196bc132e6e504f12 => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:53.670321 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba1a22-60a9-4a6f-ad89-91c52ba64e5c None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:53.689203 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33815f61-9cb0-4a0d-b764-72111e5b1d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:53.690460 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33815f61-9cb0-4a0d-b764-72111e5b1d33 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:53.690919 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33815f61-9cb0-4a0d-b764-72111e5b1d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:53.691439 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33815f61-9cb0-4a0d-b764-72111e5b1d33 None admin] PATH_INFO: `/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:53.692926 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33815f61-9cb0-4a0d-b764-72111e5b1d33 None admin] RBAC: Authorizing `identity:get_project(project_id=a6fde692f8db4d7eb4858942b0b6e5bf)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:53.694634 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33815f61-9cb0-4a0d-b764-72111e5b1d33 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:53.696837 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 112/226] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:14:53 2022] GET /identity/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:53.702588 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33815f61-9cb0-4a0d-b764-72111e5b1d33 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:53.717335 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:53.781034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba1a22-60a9-4a6f-ad89-91c52ba64e5c None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:53.819912 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfaad1e-18fb-4e3e-8d4f-56acd66ca6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:53.821716 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfaad1e-18fb-4e3e-8d4f-56acd66ca6a6 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:53.822183 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfaad1e-18fb-4e3e-8d4f-56acd66ca6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:53.822869 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfaad1e-18fb-4e3e-8d4f-56acd66ca6a6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:53.830357 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33815f61-9cb0-4a0d-b764-72111e5b1d33 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:53.837667 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfaad1e-18fb-4e3e-8d4f-56acd66ca6a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:14:53.850513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-cdfaad1e-18fb-4e3e-8d4f-56acd66ca6a6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 15:14:53.850513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:14:53.850513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:14:53.850513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:14:53.850513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:14:53.850513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:14:53.850513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:14:53.850513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:14:53.850513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:14:53.850513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:14:53.850513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:14:53.850513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:14:53.850513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:14:53.850513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:14:53.850513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:14:53.850513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:14:53.851450 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:14:53.851450 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:14:53.851450 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:14:53.851450 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:14:53.851450 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:14:53.851450 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:14:53.851450 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:14:53.851450 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:14:53.851450 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:14:53.851450 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:14:53.851450 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:14:53.851450 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:14:53.851450 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:14:53.851450 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:14:53.851450 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:14:53.851450 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:14:53.852491 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:14:53.852491 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:14:53.852491 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:14:53.852491 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:14:53.852491 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:14:53.852491 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:14:53.852491 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:14:53.852491 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:14:53.852491 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:14:53.852491 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:14:53.852491 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 15:14:53.852491 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:14:53.855613 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 115/227] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Jul 7 15:14:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:14:53.857336 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfaad1e-18fb-4e3e-8d4f-56acd66ca6a6 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:53.872166 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fca672e-5226-4bc4-a87a-32a1d8220ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:53.873059 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca672e-5226-4bc4-a87a-32a1d8220ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:53.873346 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca672e-5226-4bc4-a87a-32a1d8220ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:53.873628 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca672e-5226-4bc4-a87a-32a1d8220ad8 None admin] PATH_INFO: `/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:53.875129 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fca672e-5226-4bc4-a87a-32a1d8220ad8 None admin] RBAC: Authorizing `identity:get_project(project_id=a6fde692f8db4d7eb4858942b0b6e5bf)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:53.877437 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fca672e-5226-4bc4-a87a-32a1d8220ad8 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:53.879372 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 113/228] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:14:53 2022] GET /identity/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:53.880572 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fca672e-5226-4bc4-a87a-32a1d8220ad8 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:53.937517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:53.984373 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:53.991638 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfaad1e-18fb-4e3e-8d4f-56acd66ca6a6 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:54.014362 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fca672e-5226-4bc4-a87a-32a1d8220ad8 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:54.033470 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7123d9-f396-415b-a9a0-bd0d11e1a03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:54.034908 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7123d9-f396-415b-a9a0-bd0d11e1a03c None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:54.035365 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7123d9-f396-415b-a9a0-bd0d11e1a03c None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:54.035827 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7123d9-f396-415b-a9a0-bd0d11e1a03c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:54.036803 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7123d9-f396-415b-a9a0-bd0d11e1a03c None admin] RBAC: Adding query filter params (role.id=d43d0d189e254500875f5f86e6abd178, scope.project.id=a6fde692f8db4d7eb4858942b0b6e5bf, user.id=7c8dca7248a54203bfb90013cf4fac8d) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:54.037259 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7123d9-f396-415b-a9a0-bd0d11e1a03c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:54.039742 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7123d9-f396-415b-a9a0-bd0d11e1a03c None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:54.048037 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 116/229] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:14:53 2022] GET /identity/v3/role_assignments?user.id=7c8dca7248a54203bfb90013cf4fac8d&scope.project.id=a6fde692f8db4d7eb4858942b0b6e5bf&role.id=d43d0d189e254500875f5f86e6abd178 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:54.049069 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7123d9-f396-415b-a9a0-bd0d11e1a03c None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:54.084339 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce72a6-e3b3-4b78-8f6e-461e5394860a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:54.085035 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce72a6-e3b3-4b78-8f6e-461e5394860a None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:54.085316 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce72a6-e3b3-4b78-8f6e-461e5394860a None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:54.085602 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce72a6-e3b3-4b78-8f6e-461e5394860a None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:54.086374 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce72a6-e3b3-4b78-8f6e-461e5394860a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:54.086687 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce72a6-e3b3-4b78-8f6e-461e5394860a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:54.088114 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce72a6-e3b3-4b78-8f6e-461e5394860a None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:54.114016 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 114/230] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:14:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:54.120668 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce72a6-e3b3-4b78-8f6e-461e5394860a None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:54.168668 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7123d9-f396-415b-a9a0-bd0d11e1a03c None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:54.200102 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:54.209939 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575fb933-35b5-4332-8fb3-0e95fcd9eaee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:54.211173 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575fb933-35b5-4332-8fb3-0e95fcd9eaee None admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:54.212083 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575fb933-35b5-4332-8fb3-0e95fcd9eaee None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:54.212524 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575fb933-35b5-4332-8fb3-0e95fcd9eaee None admin] PATH_INFO: `/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf/users/8ccc20f4d89b4c7196bc132e6e504f12/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:54.215142 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575fb933-35b5-4332-8fb3-0e95fcd9eaee None admin] RBAC: Authorizing `identity:create_grant(project_id=a6fde692f8db4d7eb4858942b0b6e5bf, user_id=8ccc20f4d89b4c7196bc132e6e504f12, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:54.217100 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575fb933-35b5-4332-8fb3-0e95fcd9eaee None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:54.312830 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce72a6-e3b3-4b78-8f6e-461e5394860a None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:54.341828 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 117/231] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Jul 7 15:14:54 2022] PUT /identity/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf/users/8ccc20f4d89b4c7196bc132e6e504f12/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:14:54.352719 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb3612-15db-4db8-b399-403f034ce9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:54.353722 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb3612-15db-4db8-b399-403f034ce9fb None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:54.354015 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb3612-15db-4db8-b399-403f034ce9fb None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:54.357771 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb3612-15db-4db8-b399-403f034ce9fb None admin] PATH_INFO: `/v3/users/8ccc20f4d89b4c7196bc132e6e504f12` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:54.357771 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb3612-15db-4db8-b399-403f034ce9fb None admin] RBAC: Authorizing `identity:get_user(user_id=8ccc20f4d89b4c7196bc132e6e504f12)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:54.357771 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb3612-15db-4db8-b399-403f034ce9fb None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:54.359512 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 115/232] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:14:54 2022] GET /identity/v3/users/8ccc20f4d89b4c7196bc132e6e504f12 => generated 283 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:54.364926 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575fb933-35b5-4332-8fb3-0e95fcd9eaee None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:54.491233 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575fb933-35b5-4332-8fb3-0e95fcd9eaee None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:54.527709 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f493c77-3bb9-4726-805f-140a24b2d498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:54.528631 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f493c77-3bb9-4726-805f-140a24b2d498 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:54.528917 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f493c77-3bb9-4726-805f-140a24b2d498 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:54.529194 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f493c77-3bb9-4726-805f-140a24b2d498 None admin] PATH_INFO: `/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:54.530411 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f493c77-3bb9-4726-805f-140a24b2d498 None admin] RBAC: Authorizing `identity:get_project(project_id=a6fde692f8db4d7eb4858942b0b6e5bf)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:54.531904 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f493c77-3bb9-4726-805f-140a24b2d498 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:54.533961 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 118/233] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:14:54 2022] GET /identity/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:54.539596 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb3612-15db-4db8-b399-403f034ce9fb None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:54.553056 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:54.658685 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb3612-15db-4db8-b399-403f034ce9fb None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:54.695902 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12316436-642f-4ee8-9f26-2973124b54cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:54.697022 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12316436-642f-4ee8-9f26-2973124b54cd None admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:54.697442 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12316436-642f-4ee8-9f26-2973124b54cd None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:54.697873 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12316436-642f-4ee8-9f26-2973124b54cd None admin] PATH_INFO: `/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf/users/8ccc20f4d89b4c7196bc132e6e504f12/roles/c82203e4734346719cafb16230f45942` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:54.700379 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12316436-642f-4ee8-9f26-2973124b54cd None admin] RBAC: Authorizing `identity:create_grant(project_id=a6fde692f8db4d7eb4858942b0b6e5bf, user_id=8ccc20f4d89b4c7196bc132e6e504f12, role_id=c82203e4734346719cafb16230f45942)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:54.701951 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12316436-642f-4ee8-9f26-2973124b54cd None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:54.809082 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 116/234] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Jul 7 15:14:54 2022] PUT /identity/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf/users/8ccc20f4d89b4c7196bc132e6e504f12/roles/c82203e4734346719cafb16230f45942 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:14:55.380590 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f934e-f846-477d-8a3d-8f08c690122d None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:55.381114 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f934e-f846-477d-8a3d-8f08c690122d None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:55.381402 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f934e-f846-477d-8a3d-8f08c690122d None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:55.383025 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61c7d8-3c03-41ed-98c6-b003e6638647 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:55.383406 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61c7d8-3c03-41ed-98c6-b003e6638647 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:55.385116 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 119/235] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:14:55.385391 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61c7d8-3c03-41ed-98c6-b003e6638647 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:55.386887 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 117/236] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:14:55.391768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b1e9d-868d-4f7f-b792-6dc2b214ac45 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:55.392655 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b1e9d-868d-4f7f-b792-6dc2b214ac45 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:55.392888 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b1e9d-868d-4f7f-b792-6dc2b214ac45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:55.395637 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6e867-2261-48dc-b16b-615faeaae941 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:55.395922 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6e867-2261-48dc-b16b-615faeaae941 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:55.396199 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6e867-2261-48dc-b16b-615faeaae941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:55.425880 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-2d1b1e9d-868d-4f7f-b792-6dc2b214ac45 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:55.427086 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-2d1b1e9d-868d-4f7f-b792-6dc2b214ac45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:55.429916 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-4fc6e867-2261-48dc-b16b-615faeaae941 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:55.431079 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-4fc6e867-2261-48dc-b16b-615faeaae941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:55.526769 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:55.536969 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:55.547938 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 120/237] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:14:55.555838 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 118/238] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:14:55.576263 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b1e9d-868d-4f7f-b792-6dc2b214ac45 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:55.592924 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6e867-2261-48dc-b16b-615faeaae941 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:55.715375 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b1e9d-868d-4f7f-b792-6dc2b214ac45 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:55.754465 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c154f313-511c-4222-be8d-daa4a06a5f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:55.755256 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154f313-511c-4222-be8d-daa4a06a5f82 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:55.755536 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154f313-511c-4222-be8d-daa4a06a5f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:55.755834 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154f313-511c-4222-be8d-daa4a06a5f82 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:55.757110 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c154f313-511c-4222-be8d-daa4a06a5f82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:55.758557 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c154f313-511c-4222-be8d-daa4a06a5f82 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:55.760597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 121/239] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:14:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:55.765889 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c154f313-511c-4222-be8d-daa4a06a5f82 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:55.775254 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6e867-2261-48dc-b16b-615faeaae941 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:55.818826 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:55.868952 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3876306c-9c5e-444a-8c9e-56d5d38e5b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:55.869771 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876306c-9c5e-444a-8c9e-56d5d38e5b12 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:55.870059 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876306c-9c5e-444a-8c9e-56d5d38e5b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:55.870335 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876306c-9c5e-444a-8c9e-56d5d38e5b12 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:55.873863 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c154f313-511c-4222-be8d-daa4a06a5f82 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:55.906303 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3876306c-9c5e-444a-8c9e-56d5d38e5b12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:14:55.916862 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-3876306c-9c5e-444a-8c9e-56d5d38e5b12 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 07 15:14:55.916862 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:14:55.916862 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:14:55.916862 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:14:55.916862 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:14:55.916862 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:14:55.916862 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:14:55.916862 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:14:55.916862 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:14:55.916862 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:14:55.916862 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:14:55.916862 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:14:55.916862 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:14:55.916862 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:14:55.916862 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:14:55.916862 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:14:55.917767 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:14:55.917767 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:14:55.917767 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:14:55.917767 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:14:55.917767 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:14:55.917767 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:14:55.917767 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:14:55.917767 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:14:55.917767 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:14:55.917767 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:14:55.917767 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:14:55.917767 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:14:55.917767 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:14:55.917767 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:14:55.917767 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:14:55.917767 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:14:55.918993 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:14:55.918993 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:14:55.918993 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:14:55.918993 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:14:55.918993 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:14:55.918993 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:14:55.918993 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:14:55.918993 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:14:55.918993 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:14:55.918993 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:14:55.918993 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 07 15:14:55.918993 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:14:55.918993 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 119/240] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:14:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:14:55.920370 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db8778-83f6-4000-8997-698451564e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:55.921272 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db8778-83f6-4000-8997-698451564e28 None admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:55.921748 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db8778-83f6-4000-8997-698451564e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:55.922206 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db8778-83f6-4000-8997-698451564e28 None admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:55.924594 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35db8778-83f6-4000-8997-698451564e28 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:55.924703 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3876306c-9c5e-444a-8c9e-56d5d38e5b12 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:55.926404 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35db8778-83f6-4000-8997-698451564e28 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:56.001146 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:56.037875 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 122/241] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Jul 7 15:14:55 2022] POST /identity/v3/users => generated 293 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:14:56.053580 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3876306c-9c5e-444a-8c9e-56d5d38e5b12 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:56.090987 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c16c114-5a9e-4caa-a4d6-5d277fa36ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:56.091731 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16c114-5a9e-4caa-a4d6-5d277fa36ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:56.092073 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16c114-5a9e-4caa-a4d6-5d277fa36ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:56.092346 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16c114-5a9e-4caa-a4d6-5d277fa36ec6 None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:56.093113 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c16c114-5a9e-4caa-a4d6-5d277fa36ec6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:56.093415 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c16c114-5a9e-4caa-a4d6-5d277fa36ec6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:56.094790 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c16c114-5a9e-4caa-a4d6-5d277fa36ec6 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:56.105650 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 120/242] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Jul 7 15:14:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:56.111814 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db8778-83f6-4000-8997-698451564e28 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:56.241487 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db8778-83f6-4000-8997-698451564e28 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:56.244309 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:56.292274 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796064e-7bc3-49a1-98a7-9bb56997bef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:56.293087 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796064e-7bc3-49a1-98a7-9bb56997bef0 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:56.293370 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796064e-7bc3-49a1-98a7-9bb56997bef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:56.293639 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796064e-7bc3-49a1-98a7-9bb56997bef0 None admin] PATH_INFO: `/v3/users/ab5f8b8c8d574748aa17a300aaf6e2db` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:56.325916 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0796064e-7bc3-49a1-98a7-9bb56997bef0 None admin] RBAC: Authorizing `identity:get_user(user_id=ab5f8b8c8d574748aa17a300aaf6e2db)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:56.327415 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0796064e-7bc3-49a1-98a7-9bb56997bef0 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:56.329442 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 123/243] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:14:56 2022] GET /identity/v3/users/ab5f8b8c8d574748aa17a300aaf6e2db => generated 297 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:56.335453 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c16c114-5a9e-4caa-a4d6-5d277fa36ec6 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:56.398862 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bea82c-c1c9-4cc9-9fd0-2d4b07d1367c None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:56.399398 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bea82c-c1c9-4cc9-9fd0-2d4b07d1367c None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:56.399398 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bea82c-c1c9-4cc9-9fd0-2d4b07d1367c None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:56.400860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 124/244] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:14:56.406164 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187b98b-3610-4bb1-9a13-34019d8690da None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:56.406450 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187b98b-3610-4bb1-9a13-34019d8690da None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:56.406723 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187b98b-3610-4bb1-9a13-34019d8690da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:56.433199 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-2187b98b-3610-4bb1-9a13-34019d8690da None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:56.434392 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-2187b98b-3610-4bb1-9a13-34019d8690da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:56.471019 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c16c114-5a9e-4caa-a4d6-5d277fa36ec6 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:56.492953 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:56.514229 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbb95ab-7765-428c-83aa-3e16e1245951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:56.515284 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb95ab-7765-428c-83aa-3e16e1245951 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:56.515781 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb95ab-7765-428c-83aa-3e16e1245951 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:56.516236 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb95ab-7765-428c-83aa-3e16e1245951 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:56.535336 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 125/245] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 15:14:56.560269 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2187b98b-3610-4bb1-9a13-34019d8690da None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:56.608720 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbb95ab-7765-428c-83aa-3e16e1245951 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:56.608720 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbb95ab-7765-428c-83aa-3e16e1245951 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:56.727088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 121/246] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:14:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 15:14:56.736618 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:56.737718 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbb95ab-7765-428c-83aa-3e16e1245951 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:56.743410 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2187b98b-3610-4bb1-9a13-34019d8690da None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:56.758074 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:56.781975 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eec8c0-aac8-454c-9712-4be8c6b8c74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:56.782796 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eec8c0-aac8-454c-9712-4be8c6b8c74f None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:56.783080 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eec8c0-aac8-454c-9712-4be8c6b8c74f None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:56.783360 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eec8c0-aac8-454c-9712-4be8c6b8c74f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:56.794171 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eec8c0-aac8-454c-9712-4be8c6b8c74f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:14:56.802625 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-95eec8c0-aac8-454c-9712-4be8c6b8c74f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 07 15:14:56.802625 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:14:56.802625 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:14:56.802625 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:14:56.802625 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:14:56.802625 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:14:56.802625 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:14:56.802625 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:14:56.802625 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:14:56.802625 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:14:56.802625 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:14:56.802625 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:14:56.802625 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:14:56.802625 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:14:56.802625 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:14:56.802625 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:14:56.803766 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:14:56.803766 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:14:56.803766 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:14:56.803766 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:14:56.803766 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:14:56.803766 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:14:56.803766 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:14:56.803766 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:14:56.803766 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:14:56.803766 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:14:56.803766 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:14:56.803766 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:14:56.803766 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:14:56.803766 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:14:56.803766 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:14:56.803766 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:14:56.804737 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:14:56.804737 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:14:56.804737 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:14:56.804737 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:14:56.804737 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:14:56.804737 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:14:56.804737 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:14:56.804737 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:14:56.804737 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:14:56.804737 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:14:56.804737 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 07 15:14:56.804737 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:14:56.805422 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 126/247] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:14:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:14:56.814421 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eec8c0-aac8-454c-9712-4be8c6b8c74f None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:56.859202 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbb95ab-7765-428c-83aa-3e16e1245951 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:56.898710 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b6144-f529-46f6-9ce2-8334f3a63666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:56.899844 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b6144-f529-46f6-9ce2-8334f3a63666 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:56.900263 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b6144-f529-46f6-9ce2-8334f3a63666 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:56.900693 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b6144-f529-46f6-9ce2-8334f3a63666 None admin] PATH_INFO: `/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:56.902065 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12b6144-f529-46f6-9ce2-8334f3a63666 None admin] RBAC: Authorizing `identity:get_project(project_id=a6fde692f8db4d7eb4858942b0b6e5bf)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:56.903696 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12b6144-f529-46f6-9ce2-8334f3a63666 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:56.905575 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 122/248] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:14:56 2022] GET /identity/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:56.906944 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97bbef-e794-44da-addd-0091eca3038f None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:56.907398 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97bbef-e794-44da-addd-0091eca3038f None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:56.907875 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97bbef-e794-44da-addd-0091eca3038f None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:56.909258 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 123/249] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:14:56.914156 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97bbef-e794-44da-addd-0091eca3038f None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:56.952170 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eec8c0-aac8-454c-9712-4be8c6b8c74f None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:57.023942 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59566f-e67f-4321-8ad1-5dfb74a98f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:57.024846 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59566f-e67f-4321-8ad1-5dfb74a98f1c None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:57.025129 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59566f-e67f-4321-8ad1-5dfb74a98f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:57.025403 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59566f-e67f-4321-8ad1-5dfb74a98f1c None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:57.027135 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f59566f-e67f-4321-8ad1-5dfb74a98f1c None admin] RBAC: Adding query filter params (name=member) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:57.027452 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f59566f-e67f-4321-8ad1-5dfb74a98f1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:57.032066 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f59566f-e67f-4321-8ad1-5dfb74a98f1c None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:57.050916 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 127/250] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Jul 7 15:14:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:57.052967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f885dc5-8bfa-4efa-8c39-df3312a7edbd None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:57.053257 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f885dc5-8bfa-4efa-8c39-df3312a7edbd None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:57.053532 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f885dc5-8bfa-4efa-8c39-df3312a7edbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:57.089513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-2f885dc5-8bfa-4efa-8c39-df3312a7edbd None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:57.090720 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-2f885dc5-8bfa-4efa-8c39-df3312a7edbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:57.095977 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:57.110170 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:57.130001 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97bbef-e794-44da-addd-0091eca3038f None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:57.202398 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7facae79-bc2e-4736-a7b5-35d2b27593e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:57.203066 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7facae79-bc2e-4736-a7b5-35d2b27593e6 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:57.203350 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7facae79-bc2e-4736-a7b5-35d2b27593e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:57.203731 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7facae79-bc2e-4736-a7b5-35d2b27593e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:57.204677 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7facae79-bc2e-4736-a7b5-35d2b27593e6 None admin] RBAC: Adding query filter params (role.id=4fe04bd6b66542379945fab86d689ae6, scope.project.id=a6fde692f8db4d7eb4858942b0b6e5bf, user.id=ab5f8b8c8d574748aa17a300aaf6e2db) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:57.204993 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7facae79-bc2e-4736-a7b5-35d2b27593e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:57.208187 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7facae79-bc2e-4736-a7b5-35d2b27593e6 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:57.215726 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 124/251] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:14:56 2022] GET /identity/v3/role_assignments?user.id=ab5f8b8c8d574748aa17a300aaf6e2db&scope.project.id=a6fde692f8db4d7eb4858942b0b6e5bf&role.id=4fe04bd6b66542379945fab86d689ae6 => generated 259 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:57.216676 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7facae79-bc2e-4736-a7b5-35d2b27593e6 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:57.228626 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 128/252] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:14:57.266086 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f885dc5-8bfa-4efa-8c39-df3312a7edbd None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:57.410192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7facae79-bc2e-4736-a7b5-35d2b27593e6 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:57.496349 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1195671-a310-40b6-86bb-49f98c5e1792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:57.497370 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1195671-a310-40b6-86bb-49f98c5e1792 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:57.497721 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1195671-a310-40b6-86bb-49f98c5e1792 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:57.498066 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1195671-a310-40b6-86bb-49f98c5e1792 None admin] PATH_INFO: `/v3/users/7c8dca7248a54203bfb90013cf4fac8d` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:57.499387 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1195671-a310-40b6-86bb-49f98c5e1792 None admin] RBAC: Authorizing `identity:get_user(user_id=7c8dca7248a54203bfb90013cf4fac8d)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:57.501015 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1195671-a310-40b6-86bb-49f98c5e1792 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:57.507220 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 125/253] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:14:57 2022] GET /identity/v3/users/7c8dca7248a54203bfb90013cf4fac8d => generated 297 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:57.508423 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1195671-a310-40b6-86bb-49f98c5e1792 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:57.529010 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f885dc5-8bfa-4efa-8c39-df3312a7edbd None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:57.542414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:57.616696 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df4dbf0-bc3b-4694-8401-c91eda1708d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:57.617804 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4dbf0-bc3b-4694-8401-c91eda1708d3 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:57.618297 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4dbf0-bc3b-4694-8401-c91eda1708d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:57.618747 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4dbf0-bc3b-4694-8401-c91eda1708d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:57.645650 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df4dbf0-bc3b-4694-8401-c91eda1708d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:14:57.664431 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-4df4dbf0-bc3b-4694-8401-c91eda1708d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:14:57.664431 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:14:57.664431 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:14:57.664431 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:14:57.664431 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:14:57.664431 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:14:57.664431 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:14:57.664431 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:14:57.664431 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:14:57.664431 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:14:57.664431 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:14:57.664431 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:14:57.664431 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:14:57.664431 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:14:57.664431 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:14:57.664431 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:14:57.665350 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:14:57.665350 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:14:57.665350 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:14:57.665350 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:14:57.665350 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:14:57.665350 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:14:57.665350 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:14:57.665350 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:14:57.665350 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:14:57.665350 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:14:57.665350 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:14:57.665350 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:14:57.665350 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:14:57.665350 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:14:57.665350 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:14:57.665350 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:14:57.666152 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:14:57.666152 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:14:57.666152 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:14:57.666152 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:14:57.666152 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:14:57.666152 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:14:57.666152 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:14:57.666152 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:14:57.666152 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:14:57.666152 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:14:57.666152 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:14:57.666152 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:14:57.669093 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 129/254] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Jul 7 15:14:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 403 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:14:57.675051 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df4dbf0-bc3b-4694-8401-c91eda1708d3 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:57.683093 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:57.719486 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1195671-a310-40b6-86bb-49f98c5e1792 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:57.808506 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe3d03-29fc-4752-8121-b89488c22026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:57.809744 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe3d03-29fc-4752-8121-b89488c22026 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:57.811155 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe3d03-29fc-4752-8121-b89488c22026 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:57.811798 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe3d03-29fc-4752-8121-b89488c22026 None admin] PATH_INFO: `/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:57.814343 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffe3d03-29fc-4752-8121-b89488c22026 None admin] RBAC: Authorizing `identity:get_project(project_id=a6fde692f8db4d7eb4858942b0b6e5bf)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:57.817478 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffe3d03-29fc-4752-8121-b89488c22026 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:57.819533 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 126/255] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:14:57 2022] GET /identity/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf => generated 302 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:14:57.822108 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3723c6e3-194c-455f-99e2-cabd7a849bad None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:57.823336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3723c6e3-194c-455f-99e2-cabd7a849bad None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:57.824437 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3723c6e3-194c-455f-99e2-cabd7a849bad None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:57.828293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 127/256] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:14:57.829160 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3723c6e3-194c-455f-99e2-cabd7a849bad None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:57.833080 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df4dbf0-bc3b-4694-8401-c91eda1708d3 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:57.898981 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5381f1-2cf0-4145-808d-698354031777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:57.900205 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5381f1-2cf0-4145-808d-698354031777 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:57.900674 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5381f1-2cf0-4145-808d-698354031777 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:57.901204 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5381f1-2cf0-4145-808d-698354031777 None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:57.907637 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5381f1-2cf0-4145-808d-698354031777 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:57.907637 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5381f1-2cf0-4145-808d-698354031777 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:57.907637 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5381f1-2cf0-4145-808d-698354031777 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:57.913869 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 130/257] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:14:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:14:57.915329 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce4706-2b6d-4c06-9c1c-65f23aead5ee None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:57.915837 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce4706-2b6d-4c06-9c1c-65f23aead5ee None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:57.916290 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce4706-2b6d-4c06-9c1c-65f23aead5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:57.973737 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-9fce4706-2b6d-4c06-9c1c-65f23aead5ee None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:57.977362 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-9fce4706-2b6d-4c06-9c1c-65f23aead5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:58.036672 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:58.050493 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3723c6e3-194c-455f-99e2-cabd7a849bad None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:58.093138 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a3a2f0-0bfd-4ac6-adb5-f2258c08ddbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:58.094375 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a3a2f0-0bfd-4ac6-adb5-f2258c08ddbd None admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:58.094725 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a3a2f0-0bfd-4ac6-adb5-f2258c08ddbd None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:58.095111 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a3a2f0-0bfd-4ac6-adb5-f2258c08ddbd None admin] PATH_INFO: `/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf/users/7c8dca7248a54203bfb90013cf4fac8d/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:58.098214 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a3a2f0-0bfd-4ac6-adb5-f2258c08ddbd None admin] RBAC: Authorizing `identity:create_grant(project_id=a6fde692f8db4d7eb4858942b0b6e5bf, user_id=7c8dca7248a54203bfb90013cf4fac8d, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:58.099674 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:58.101572 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a3a2f0-0bfd-4ac6-adb5-f2258c08ddbd None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:58.114026 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 131/258] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:14:58.114667 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce4706-2b6d-4c06-9c1c-65f23aead5ee None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:58.183418 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 128/259] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Jul 7 15:14:57 2022] PUT /identity/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf/users/7c8dca7248a54203bfb90013cf4fac8d/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:14:58.185318 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a3a2f0-0bfd-4ac6-adb5-f2258c08ddbd None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:58.262728 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce4706-2b6d-4c06-9c1c-65f23aead5ee None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:58.318858 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a3a2f0-0bfd-4ac6-adb5-f2258c08ddbd None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:58.356361 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53720381-1e42-4c40-adf9-2f5c36504ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:58.357393 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53720381-1e42-4c40-adf9-2f5c36504ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:58.357637 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53720381-1e42-4c40-adf9-2f5c36504ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:58.357913 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53720381-1e42-4c40-adf9-2f5c36504ed6 None admin] PATH_INFO: `/v3/users/8ccc20f4d89b4c7196bc132e6e504f12` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:58.361911 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681b7d7-8811-415d-ae19-c397b83be41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:58.362949 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681b7d7-8811-415d-ae19-c397b83be41e None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:58.363425 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681b7d7-8811-415d-ae19-c397b83be41e None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:58.363977 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681b7d7-8811-415d-ae19-c397b83be41e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:58.364290 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53720381-1e42-4c40-adf9-2f5c36504ed6 None admin] RBAC: Authorizing `identity:get_user(user_id=8ccc20f4d89b4c7196bc132e6e504f12)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:58.365767 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53720381-1e42-4c40-adf9-2f5c36504ed6 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:58.367895 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 132/260] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:14:58 2022] GET /identity/v3/users/8ccc20f4d89b4c7196bc132e6e504f12 => generated 283 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:58.373205 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53720381-1e42-4c40-adf9-2f5c36504ed6 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:58.378139 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e681b7d7-8811-415d-ae19-c397b83be41e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:14:58.388145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-e681b7d7-8811-415d-ae19-c397b83be41e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 15:14:58.388145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:14:58.388145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:14:58.388145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:14:58.388145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:14:58.388145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:14:58.388145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:14:58.388145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:14:58.388145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:14:58.388145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:14:58.388145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:14:58.388145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:14:58.388145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:14:58.388145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:14:58.388145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:14:58.388145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:14:58.389109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:14:58.389109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:14:58.389109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:14:58.389109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:14:58.389109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:14:58.389109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:14:58.389109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:14:58.389109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:14:58.389109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:14:58.389109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:14:58.389109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:14:58.389109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:14:58.389109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:14:58.389109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:14:58.389109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:14:58.389109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:14:58.389924 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:14:58.389924 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:14:58.389924 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:14:58.389924 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:14:58.389924 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:14:58.389924 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:14:58.389924 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:14:58.389924 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:14:58.389924 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:14:58.389924 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:14:58.389924 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 15:14:58.389924 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:14:58.393597 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 129/261] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Jul 7 15:14:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:14:58.399734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681b7d7-8811-415d-ae19-c397b83be41e None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:58.420780 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:58.517294 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:58.603070 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681b7d7-8811-415d-ae19-c397b83be41e None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:58.607504 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53720381-1e42-4c40-adf9-2f5c36504ed6 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:58.671671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d3d2f-147d-4c60-9078-dd5c3135e210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:58.672354 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d3d2f-147d-4c60-9078-dd5c3135e210 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:58.672642 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d3d2f-147d-4c60-9078-dd5c3135e210 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:58.672911 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d3d2f-147d-4c60-9078-dd5c3135e210 None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:58.674229 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d3d2f-147d-4c60-9078-dd5c3135e210 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:58.674542 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d3d2f-147d-4c60-9078-dd5c3135e210 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:58.675963 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d3d2f-147d-4c60-9078-dd5c3135e210 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:58.685732 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619fe76b-5fbc-4b79-9dd2-0950b7820190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:58.686401 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619fe76b-5fbc-4b79-9dd2-0950b7820190 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:58.686685 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619fe76b-5fbc-4b79-9dd2-0950b7820190 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:58.686960 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619fe76b-5fbc-4b79-9dd2-0950b7820190 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:58.697876 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 130/262] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:14:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:58.719345 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d3d2f-147d-4c60-9078-dd5c3135e210 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:58.767111 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619fe76b-5fbc-4b79-9dd2-0950b7820190 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:58.769074 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619fe76b-5fbc-4b79-9dd2-0950b7820190 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:58.836557 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:58.879629 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 133/263] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:14:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 15:14:58.884760 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619fe76b-5fbc-4b79-9dd2-0950b7820190 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:58.926005 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:58.968006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d3d2f-147d-4c60-9078-dd5c3135e210 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:59.010288 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2000ae-b694-484d-9ca0-2d5a368fc9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:59.011173 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2000ae-b694-484d-9ca0-2d5a368fc9a5 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:59.011458 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2000ae-b694-484d-9ca0-2d5a368fc9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:59.011764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2000ae-b694-484d-9ca0-2d5a368fc9a5 None admin] PATH_INFO: `/v3/users/8ccc20f4d89b4c7196bc132e6e504f12` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:59.015613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2000ae-b694-484d-9ca0-2d5a368fc9a5 None admin] RBAC: Authorizing `identity:get_user(user_id=8ccc20f4d89b4c7196bc132e6e504f12)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:59.015613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2000ae-b694-484d-9ca0-2d5a368fc9a5 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:59.016691 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 131/264] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:14:58 2022] GET /identity/v3/users/8ccc20f4d89b4c7196bc132e6e504f12 => generated 283 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:14:59.023306 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2000ae-b694-484d-9ca0-2d5a368fc9a5 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:59.097228 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619fe76b-5fbc-4b79-9dd2-0950b7820190 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:59.168131 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71622361-823d-4c2a-aefe-b096f34d18e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:59.169556 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71622361-823d-4c2a-aefe-b096f34d18e7 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:59.170830 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71622361-823d-4c2a-aefe-b096f34d18e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:59.171310 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71622361-823d-4c2a-aefe-b096f34d18e7 None admin] PATH_INFO: `/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:59.178219 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71622361-823d-4c2a-aefe-b096f34d18e7 None admin] RBAC: Authorizing `identity:get_project(project_id=a6fde692f8db4d7eb4858942b0b6e5bf)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:59.179957 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71622361-823d-4c2a-aefe-b096f34d18e7 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:59.181943 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 134/265] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:14:58 2022] GET /identity/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf => generated 302 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:14:59.183815 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec1bea-cac6-489e-b255-c904ff629013 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:59.185946 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec1bea-cac6-489e-b255-c904ff629013 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:59.186602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec1bea-cac6-489e-b255-c904ff629013 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:59.189376 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 135/266] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:14:59.191292 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aec1bea-cac6-489e-b255-c904ff629013 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:59.226983 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2000ae-b694-484d-9ca0-2d5a368fc9a5 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:59.236215 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:59.272925 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a323c87-8ee7-4620-8f25-240d210ac6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:59.273637 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a323c87-8ee7-4620-8f25-240d210ac6bf None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:59.273864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a323c87-8ee7-4620-8f25-240d210ac6bf None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:59.274140 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a323c87-8ee7-4620-8f25-240d210ac6bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:59.280095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:59.329187 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aec1bea-cac6-489e-b255-c904ff629013 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:59.349178 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a323c87-8ee7-4620-8f25-240d210ac6bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:59.350662 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a323c87-8ee7-4620-8f25-240d210ac6bf None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:59.374038 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2982af9-08f9-4234-8558-ff3e685a2e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:59.375711 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2982af9-08f9-4234-8558-ff3e685a2e19 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:59.376572 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2982af9-08f9-4234-8558-ff3e685a2e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:59.377054 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2982af9-08f9-4234-8558-ff3e685a2e19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:59.378782 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2982af9-08f9-4234-8558-ff3e685a2e19 None admin] RBAC: Adding query filter params (role.id=0e7793fbf3064037984da651879e3a94, scope.project.id=a6fde692f8db4d7eb4858942b0b6e5bf, user.id=8ccc20f4d89b4c7196bc132e6e504f12) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:14:59.379613 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2982af9-08f9-4234-8558-ff3e685a2e19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:59.382144 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2982af9-08f9-4234-8558-ff3e685a2e19 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:59.390076 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 136/267] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:14:59 2022] GET /identity/v3/role_assignments?user.id=8ccc20f4d89b4c7196bc132e6e504f12&scope.project.id=a6fde692f8db4d7eb4858942b0b6e5bf&role.id=0e7793fbf3064037984da651879e3a94 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:59.392697 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d753ef-4c24-45c2-a23a-3706587ec5d4 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:59.393637 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d753ef-4c24-45c2-a23a-3706587ec5d4 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:59.394161 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d753ef-4c24-45c2-a23a-3706587ec5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:59.428546 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-77d753ef-4c24-45c2-a23a-3706587ec5d4 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:59.429779 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-77d753ef-4c24-45c2-a23a-3706587ec5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:59.531719 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 132/268] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:14:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 07 15:14:59.533599 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b95165-10eb-42e0-b0cb-2dc4e15a853c None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:59.533880 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b95165-10eb-42e0-b0cb-2dc4e15a853c None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:59.534150 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b95165-10eb-42e0-b0cb-2dc4e15a853c None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:59.539645 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 133/269] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:14:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:14:59.541689 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b95165-10eb-42e0-b0cb-2dc4e15a853c None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:59.566019 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 137/270] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:14:59.567830 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113a61f-8215-45a9-b714-d37e9a430bb7 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:59.568896 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113a61f-8215-45a9-b714-d37e9a430bb7 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:59.569386 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113a61f-8215-45a9-b714-d37e9a430bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:59.589165 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-d113a61f-8215-45a9-b714-d37e9a430bb7 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:14:59.590665 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-d113a61f-8215-45a9-b714-d37e9a430bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:14:59.618814 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:59.624012 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:59.687966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 138/271] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:14:59.695595 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d113a61f-8215-45a9-b714-d37e9a430bb7 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:59.732867 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b95165-10eb-42e0-b0cb-2dc4e15a853c None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:59.777698 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f509982-57fa-4ac6-94b2-29da26eabe1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:59.778560 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f509982-57fa-4ac6-94b2-29da26eabe1b None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:59.778839 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f509982-57fa-4ac6-94b2-29da26eabe1b None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:59.779104 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f509982-57fa-4ac6-94b2-29da26eabe1b None admin] PATH_INFO: `/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:59.780321 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f509982-57fa-4ac6-94b2-29da26eabe1b None admin] RBAC: Authorizing `identity:get_project(project_id=a6fde692f8db4d7eb4858942b0b6e5bf)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:59.781841 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f509982-57fa-4ac6-94b2-29da26eabe1b None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:59.783704 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 134/272] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:14:59 2022] GET /identity/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:59.785912 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f509982-57fa-4ac6-94b2-29da26eabe1b None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:59.848810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d113a61f-8215-45a9-b714-d37e9a430bb7 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:59.896173 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc7777-e403-4d0d-a862-8d8a2dd4cf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:59.897816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc7777-e403-4d0d-a862-8d8a2dd4cf6b None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:59.898693 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc7777-e403-4d0d-a862-8d8a2dd4cf6b None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:59.899445 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc7777-e403-4d0d-a862-8d8a2dd4cf6b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:59.913690 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dc7777-e403-4d0d-a862-8d8a2dd4cf6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:14:59.922246 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f509982-57fa-4ac6-94b2-29da26eabe1b None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:14:59.924498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-09dc7777-e403-4d0d-a862-8d8a2dd4cf6b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 07 15:14:59.924498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:14:59.924498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:14:59.924498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:14:59.924498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:14:59.924498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:14:59.924498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:14:59.924498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:14:59.924498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:14:59.924498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:14:59.924498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:14:59.924498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:14:59.924498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:14:59.924498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:14:59.924498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:14:59.924498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:14:59.925396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:14:59.925396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:14:59.925396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:14:59.925396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:14:59.925396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:14:59.925396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:14:59.925396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:14:59.925396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:14:59.925396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:14:59.925396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:14:59.925396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:14:59.925396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:14:59.925396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:14:59.925396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:14:59.925396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:14:59.925396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:14:59.926241 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:14:59.926241 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:14:59.926241 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:14:59.926241 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:14:59.926241 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:14:59.926241 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:14:59.926241 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:14:59.926241 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:14:59.926241 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:14:59.926241 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:14:59.926241 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 07 15:14:59.926241 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:14:59.928033 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:59.929326 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 139/273] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:14:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:14:59.932086 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc7777-e403-4d0d-a862-8d8a2dd4cf6b None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:14:59.958461 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:14:59.963466 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6c3dbe-fa91-4a64-aff4-381d53bd175b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:14:59.964356 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6c3dbe-fa91-4a64-aff4-381d53bd175b None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:14:59.965026 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6c3dbe-fa91-4a64-aff4-381d53bd175b None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:14:59.965026 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6c3dbe-fa91-4a64-aff4-381d53bd175b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:14:59.966380 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6c3dbe-fa91-4a64-aff4-381d53bd175b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:14:59.968003 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6c3dbe-fa91-4a64-aff4-381d53bd175b None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:14:59.975697 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 135/274] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:14:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:14:59.975697 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6c3dbe-fa91-4a64-aff4-381d53bd175b None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:00.066195 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc7777-e403-4d0d-a862-8d8a2dd4cf6b None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:00.087242 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6c3dbe-fa91-4a64-aff4-381d53bd175b None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:00.100884 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cddf065-372e-463c-b4a7-2ff9da4075cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:00.101595 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cddf065-372e-463c-b4a7-2ff9da4075cb None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:00.101799 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cddf065-372e-463c-b4a7-2ff9da4075cb None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:00.102067 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cddf065-372e-463c-b4a7-2ff9da4075cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:00.103052 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cddf065-372e-463c-b4a7-2ff9da4075cb None admin] RBAC: Adding query filter params (role.id=c82203e4734346719cafb16230f45942, scope.project.id=a6fde692f8db4d7eb4858942b0b6e5bf, user.id=8ccc20f4d89b4c7196bc132e6e504f12) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:00.103255 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cddf065-372e-463c-b4a7-2ff9da4075cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:00.104654 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cddf065-372e-463c-b4a7-2ff9da4075cb None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:00.111724 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 140/275] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:14:59 2022] GET /identity/v3/role_assignments?user.id=8ccc20f4d89b4c7196bc132e6e504f12&scope.project.id=a6fde692f8db4d7eb4858942b0b6e5bf&role.id=c82203e4734346719cafb16230f45942 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:00.111724 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cddf065-372e-463c-b4a7-2ff9da4075cb None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:00.125858 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f8f02-0a08-460a-a90c-5b29590e0eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:00.127442 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f8f02-0a08-460a-a90c-5b29590e0eaa None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:00.127897 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f8f02-0a08-460a-a90c-5b29590e0eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:00.128322 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f8f02-0a08-460a-a90c-5b29590e0eaa None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:00.129238 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434f8f02-0a08-460a-a90c-5b29590e0eaa None admin] RBAC: Adding query filter params (name=member) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:00.129700 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434f8f02-0a08-460a-a90c-5b29590e0eaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:00.131285 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434f8f02-0a08-460a-a90c-5b29590e0eaa None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:00.141487 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 136/276] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Jul 7 15:14:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:00.143258 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ad6ef-1ff4-478b-b071-0c86df07ff90 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:00.143717 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ad6ef-1ff4-478b-b071-0c86df07ff90 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:00.144138 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ad6ef-1ff4-478b-b071-0c86df07ff90 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:00.145801 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 137/277] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:00.151270 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceeba43-bfb8-4da4-8c4b-5bf3a86994c9 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:00.151990 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceeba43-bfb8-4da4-8c4b-5bf3a86994c9 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:00.152440 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceeba43-bfb8-4da4-8c4b-5bf3a86994c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:00.178316 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-fceeba43-bfb8-4da4-8c4b-5bf3a86994c9 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:00.180591 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-fceeba43-bfb8-4da4-8c4b-5bf3a86994c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:00.220050 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:00.220515 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:00.232148 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cddf065-372e-463c-b4a7-2ff9da4075cb None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:00.272088 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90681fff-eb4b-47a6-a2c6-60efd3cd6659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:00.273036 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90681fff-eb4b-47a6-a2c6-60efd3cd6659 None admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:00.273853 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90681fff-eb4b-47a6-a2c6-60efd3cd6659 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:00.274367 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90681fff-eb4b-47a6-a2c6-60efd3cd6659 None admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:00.275444 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90681fff-eb4b-47a6-a2c6-60efd3cd6659 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:00.277986 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90681fff-eb4b-47a6-a2c6-60efd3cd6659 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:00.281488 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 138/278] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:00.287931 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceeba43-bfb8-4da4-8c4b-5bf3a86994c9 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:00.404110 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceeba43-bfb8-4da4-8c4b-5bf3a86994c9 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:00.422230 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 141/279] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Jul 7 15:15:00 2022] POST /identity/v3/users => generated 293 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:15:00.423194 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90681fff-eb4b-47a6-a2c6-60efd3cd6659 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:00.457619 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f36281-ac5f-4ac0-9cf8-11810db6bbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:00.458673 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f36281-ac5f-4ac0-9cf8-11810db6bbd4 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:00.459047 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f36281-ac5f-4ac0-9cf8-11810db6bbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:00.459447 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f36281-ac5f-4ac0-9cf8-11810db6bbd4 None admin] PATH_INFO: `/v3/users/828d69aa0f214660b3fd94c18659909d` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:00.478759 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f36281-ac5f-4ac0-9cf8-11810db6bbd4 None admin] RBAC: Authorizing `identity:get_user(user_id=828d69aa0f214660b3fd94c18659909d)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:00.480418 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f36281-ac5f-4ac0-9cf8-11810db6bbd4 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:00.487529 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 139/280] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:15:00 2022] GET /identity/v3/users/828d69aa0f214660b3fd94c18659909d => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:00.489177 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f36281-ac5f-4ac0-9cf8-11810db6bbd4 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:00.524031 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:00.553328 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90681fff-eb4b-47a6-a2c6-60efd3cd6659 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:00.589816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8412ab6f-46fa-4aad-a24d-951867142efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:00.590736 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8412ab6f-46fa-4aad-a24d-951867142efa None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:00.591018 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8412ab6f-46fa-4aad-a24d-951867142efa None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:00.591291 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8412ab6f-46fa-4aad-a24d-951867142efa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:00.594360 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:00.608425 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8412ab6f-46fa-4aad-a24d-951867142efa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:00.614427 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f36281-ac5f-4ac0-9cf8-11810db6bbd4 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:00.617472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-8412ab6f-46fa-4aad-a24d-951867142efa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 07 15:15:00.617472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:00.617472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:00.617472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:00.617472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:00.617472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:00.617472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:00.617472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:00.617472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:00.617472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:00.617472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:00.617472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:00.617472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:00.617472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:00.617472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:00.617472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:00.618341 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:00.618341 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:00.618341 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:00.618341 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:00.618341 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:00.618341 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:00.618341 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:00.618341 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:00.618341 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:00.618341 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:00.618341 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:00.618341 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:00.618341 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:00.618341 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:00.618341 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:00.618341 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:00.621576 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:00.621576 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:00.621576 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:00.621576 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:00.621576 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:00.621576 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:00.621576 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:00.621576 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:00.621576 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:00.621576 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:00.621576 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 07 15:15:00.621576 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:00.621576 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 142/281] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:15:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:00.624802 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8412ab6f-46fa-4aad-a24d-951867142efa None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:00.666403 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d68028-44c5-4c94-b4c9-126d87667389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:00.667921 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d68028-44c5-4c94-b4c9-126d87667389 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:00.668669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d68028-44c5-4c94-b4c9-126d87667389 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:00.670848 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d68028-44c5-4c94-b4c9-126d87667389 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:00.670848 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d68028-44c5-4c94-b4c9-126d87667389 None admin] RBAC: Adding query filter params (role.id=d43d0d189e254500875f5f86e6abd178, user.id=828d69aa0f214660b3fd94c18659909d, scope.system=all) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:00.670848 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d68028-44c5-4c94-b4c9-126d87667389 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:00.671961 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d68028-44c5-4c94-b4c9-126d87667389 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:00.679675 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 140/282] 10.222.0.25 () {56 vars in 1308 bytes} [Thu Jul 7 15:15:00 2022] GET /identity/v3/role_assignments?user.id=828d69aa0f214660b3fd94c18659909d&scope.system=all&role.id=d43d0d189e254500875f5f86e6abd178 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:15:00.744940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a90812-0268-4700-b34b-e43dff914038 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:00.745260 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a90812-0268-4700-b34b-e43dff914038 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:00.745495 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a90812-0268-4700-b34b-e43dff914038 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:00.747220 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 141/283] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:15:00.753196 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1dc32b-2397-4c6a-84af-b65103e45e99 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:00.753512 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1dc32b-2397-4c6a-84af-b65103e45e99 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:00.753851 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1dc32b-2397-4c6a-84af-b65103e45e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:00.756738 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8412ab6f-46fa-4aad-a24d-951867142efa None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:00.778411 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-5f1dc32b-2397-4c6a-84af-b65103e45e99 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:00.779687 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-5f1dc32b-2397-4c6a-84af-b65103e45e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:00.796677 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d50ee8f-94e8-44be-9966-6c6e83339527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:00.797358 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50ee8f-94e8-44be-9966-6c6e83339527 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:00.797644 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50ee8f-94e8-44be-9966-6c6e83339527 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:00.797924 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50ee8f-94e8-44be-9966-6c6e83339527 None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:00.798753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d50ee8f-94e8-44be-9966-6c6e83339527 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:00.799066 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d50ee8f-94e8-44be-9966-6c6e83339527 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:00.800529 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d50ee8f-94e8-44be-9966-6c6e83339527 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:00.810094 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 143/284] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Jul 7 15:15:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:15:00.816058 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d50ee8f-94e8-44be-9966-6c6e83339527 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:00.862204 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 142/285] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:00.884663 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:00.892759 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1dc32b-2397-4c6a-84af-b65103e45e99 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:00.913997 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:00.936302 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d50ee8f-94e8-44be-9966-6c6e83339527 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:00.975586 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75548b65-6305-4858-b97c-8772bea8c74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:00.975586 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75548b65-6305-4858-b97c-8772bea8c74b None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:00.976038 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75548b65-6305-4858-b97c-8772bea8c74b None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:00.976406 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75548b65-6305-4858-b97c-8772bea8c74b None admin] PATH_INFO: `/v3/users/ab5f8b8c8d574748aa17a300aaf6e2db` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:00.977745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75548b65-6305-4858-b97c-8772bea8c74b None admin] RBAC: Authorizing `identity:get_user(user_id=ab5f8b8c8d574748aa17a300aaf6e2db)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:00.979295 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75548b65-6305-4858-b97c-8772bea8c74b None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:00.981364 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 144/286] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:15:00 2022] GET /identity/v3/users/ab5f8b8c8d574748aa17a300aaf6e2db => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:00.986689 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75548b65-6305-4858-b97c-8772bea8c74b None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:01.056204 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1dc32b-2397-4c6a-84af-b65103e45e99 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:01.128410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f4177-9adc-490f-bc98-07e30c964888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:01.129219 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f4177-9adc-490f-bc98-07e30c964888 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:01.129496 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f4177-9adc-490f-bc98-07e30c964888 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:01.129768 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f4177-9adc-490f-bc98-07e30c964888 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:01.145979 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81f4177-9adc-490f-bc98-07e30c964888 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:01.155339 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-f81f4177-9adc-490f-bc98-07e30c964888 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 07 15:15:01.155339 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:01.155339 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:01.155339 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:01.155339 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:01.155339 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:01.155339 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:01.155339 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:01.155339 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:01.155339 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:01.155339 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:01.155339 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:01.155339 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:01.155339 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:01.155339 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:01.155339 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:01.156287 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:01.156287 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:01.156287 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:01.156287 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:01.156287 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:01.156287 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:01.156287 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:01.156287 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:01.156287 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:01.156287 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:01.156287 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:01.156287 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:01.156287 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:01.156287 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:01.156287 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:01.156287 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:01.157101 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:01.157101 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:01.157101 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:01.157101 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:01.157101 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:01.157101 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:01.157101 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:01.157101 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:01.157101 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:01.157101 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:01.157101 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 07 15:15:01.157101 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:01.157101 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 143/287] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:15:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:01.161655 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75548b65-6305-4858-b97c-8772bea8c74b None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:01.162348 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f4177-9adc-490f-bc98-07e30c964888 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:01.203879 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:01.214339 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be558b6-30d8-4b78-a3f6-be55fa9d8ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:01.215808 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be558b6-30d8-4b78-a3f6-be55fa9d8ba0 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:01.223709 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be558b6-30d8-4b78-a3f6-be55fa9d8ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:01.223709 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be558b6-30d8-4b78-a3f6-be55fa9d8ba0 None admin] PATH_INFO: `/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:01.223709 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be558b6-30d8-4b78-a3f6-be55fa9d8ba0 None admin] RBAC: Authorizing `identity:get_project(project_id=a6fde692f8db4d7eb4858942b0b6e5bf)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:01.223709 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be558b6-30d8-4b78-a3f6-be55fa9d8ba0 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:01.223709 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 145/288] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:15:00 2022] GET /identity/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:01.228304 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be558b6-30d8-4b78-a3f6-be55fa9d8ba0 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:01.309526 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:01.343317 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be558b6-30d8-4b78-a3f6-be55fa9d8ba0 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:01.381733 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f4177-9adc-490f-bc98-07e30c964888 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:01.382626 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e74f2-5267-4bbe-ad08-afa63f085cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:01.384014 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e74f2-5267-4bbe-ad08-afa63f085cba None admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:01.384524 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e74f2-5267-4bbe-ad08-afa63f085cba None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:01.385021 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e74f2-5267-4bbe-ad08-afa63f085cba None admin] PATH_INFO: `/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf/users/ab5f8b8c8d574748aa17a300aaf6e2db/roles/4fe04bd6b66542379945fab86d689ae6` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:01.387257 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0e74f2-5267-4bbe-ad08-afa63f085cba None admin] RBAC: Authorizing `identity:create_grant(project_id=a6fde692f8db4d7eb4858942b0b6e5bf, user_id=ab5f8b8c8d574748aa17a300aaf6e2db, role_id=4fe04bd6b66542379945fab86d689ae6)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:01.389850 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0e74f2-5267-4bbe-ad08-afa63f085cba None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:01.429617 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae534744-1371-4f98-b2e5-4326e5053a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:01.430295 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae534744-1371-4f98-b2e5-4326e5053a5f None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:01.430581 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae534744-1371-4f98-b2e5-4326e5053a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:01.430857 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae534744-1371-4f98-b2e5-4326e5053a5f None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:01.431645 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae534744-1371-4f98-b2e5-4326e5053a5f None admin] RBAC: Adding query filter params (name=member) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:01.432031 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae534744-1371-4f98-b2e5-4326e5053a5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:01.433358 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae534744-1371-4f98-b2e5-4326e5053a5f None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:01.443262 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 144/289] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Jul 7 15:15:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:01.448326 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae534744-1371-4f98-b2e5-4326e5053a5f None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:01.494109 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 146/290] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Jul 7 15:15:01 2022] PUT /identity/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf/users/ab5f8b8c8d574748aa17a300aaf6e2db/roles/4fe04bd6b66542379945fab86d689ae6 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:15:01.570358 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae534744-1371-4f98-b2e5-4326e5053a5f None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:01.608764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe1e65-3b71-4118-8d46-0c768d5215ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:01.609685 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe1e65-3b71-4118-8d46-0c768d5215ca None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:01.609974 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe1e65-3b71-4118-8d46-0c768d5215ca None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:01.610252 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe1e65-3b71-4118-8d46-0c768d5215ca None admin] PATH_INFO: `/v3/users/7c8dca7248a54203bfb90013cf4fac8d` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:01.612087 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe1e65-3b71-4118-8d46-0c768d5215ca None admin] RBAC: Authorizing `identity:get_user(user_id=7c8dca7248a54203bfb90013cf4fac8d)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:01.613523 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe1e65-3b71-4118-8d46-0c768d5215ca None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:01.615644 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 145/291] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:15:01 2022] GET /identity/v3/users/7c8dca7248a54203bfb90013cf4fac8d => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:15:01.620729 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e74f2-5267-4bbe-ad08-afa63f085cba None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:01.720498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:01.774120 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e74f2-5267-4bbe-ad08-afa63f085cba None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:01.809284 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b745c-c29f-41f3-8432-c79dedda1318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:01.809938 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b745c-c29f-41f3-8432-c79dedda1318 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:01.810222 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b745c-c29f-41f3-8432-c79dedda1318 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:01.810496 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b745c-c29f-41f3-8432-c79dedda1318 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:01.891875 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62b745c-c29f-41f3-8432-c79dedda1318 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:01.893417 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62b745c-c29f-41f3-8432-c79dedda1318 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:01.967587 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:02.036672 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 147/292] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:15:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 15:15:02.042214 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe1e65-3b71-4118-8d46-0c768d5215ca None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:02.052362 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:02.149238 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe1e65-3b71-4118-8d46-0c768d5215ca None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:02.184383 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165b88e-6fd7-4916-9868-34ce166e1851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:02.185245 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165b88e-6fd7-4916-9868-34ce166e1851 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:02.185531 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165b88e-6fd7-4916-9868-34ce166e1851 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:02.185807 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165b88e-6fd7-4916-9868-34ce166e1851 None admin] PATH_INFO: `/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:02.186996 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8165b88e-6fd7-4916-9868-34ce166e1851 None admin] RBAC: Authorizing `identity:get_project(project_id=a6fde692f8db4d7eb4858942b0b6e5bf)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:02.188504 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8165b88e-6fd7-4916-9868-34ce166e1851 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:02.190606 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 146/293] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:15:02 2022] GET /identity/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:02.196374 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b745c-c29f-41f3-8432-c79dedda1318 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:02.316280 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b745c-c29f-41f3-8432-c79dedda1318 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:02.358967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe9876f-7ccc-4e2f-bf4e-f777c0afdecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:02.359645 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9876f-7ccc-4e2f-bf4e-f777c0afdecc None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:02.359926 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9876f-7ccc-4e2f-bf4e-f777c0afdecc None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:02.360202 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9876f-7ccc-4e2f-bf4e-f777c0afdecc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:02.361074 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe9876f-7ccc-4e2f-bf4e-f777c0afdecc None admin] RBAC: Adding query filter params (role.id=d43d0d189e254500875f5f86e6abd178, scope.project.id=a6fde692f8db4d7eb4858942b0b6e5bf, user.id=7c8dca7248a54203bfb90013cf4fac8d) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:02.361381 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe9876f-7ccc-4e2f-bf4e-f777c0afdecc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:02.362797 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe9876f-7ccc-4e2f-bf4e-f777c0afdecc None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:02.369089 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 148/294] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:15:02 2022] GET /identity/v3/role_assignments?user.id=7c8dca7248a54203bfb90013cf4fac8d&scope.project.id=a6fde692f8db4d7eb4858942b0b6e5bf&role.id=d43d0d189e254500875f5f86e6abd178 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:02.858914 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582265c-fe65-47e2-9f89-72b05393a848 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:02.859197 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582265c-fe65-47e2-9f89-72b05393a848 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:02.859772 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582265c-fe65-47e2-9f89-72b05393a848 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:02.861601 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 147/295] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:02.866981 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80716b18-2725-4833-83de-730554e1753d None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:02.879007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80716b18-2725-4833-83de-730554e1753d None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:02.879007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80716b18-2725-4833-83de-730554e1753d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:02.887504 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-80716b18-2725-4833-83de-730554e1753d None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:02.888744 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-80716b18-2725-4833-83de-730554e1753d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:02.906831 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:02.978367 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 149/296] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:03.003913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582265c-fe65-47e2-9f89-72b05393a848 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:03.087817 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c9676-89ce-42d3-a55c-89d46e56d9fd None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:03.088448 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c9676-89ce-42d3-a55c-89d46e56d9fd None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:03.088925 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c9676-89ce-42d3-a55c-89d46e56d9fd None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:03.090707 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 150/297] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:03.095852 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957b6f2-af3c-4397-a495-4f41750c8d3e None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:03.096378 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:03.097193 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957b6f2-af3c-4397-a495-4f41750c8d3e None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:03.097946 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957b6f2-af3c-4397-a495-4f41750c8d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:03.116320 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-0957b6f2-af3c-4397-a495-4f41750c8d3e None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:03.117578 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-0957b6f2-af3c-4397-a495-4f41750c8d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:03.123919 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582265c-fe65-47e2-9f89-72b05393a848 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:03.174433 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1162094-2fa5-440f-b315-48fd149e713c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:03.175758 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1162094-2fa5-440f-b315-48fd149e713c None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:03.176052 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1162094-2fa5-440f-b315-48fd149e713c None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:03.176422 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1162094-2fa5-440f-b315-48fd149e713c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:03.195584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 151/298] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 15:15:03.195749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1162094-2fa5-440f-b315-48fd149e713c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:03.199075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-a1162094-2fa5-440f-b315-48fd149e713c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 07 15:15:03.199075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:03.199075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:03.199075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:03.199075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:03.199075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:03.199075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:03.199075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:03.199075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:03.199075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:03.199075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:03.199075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:03.199075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:03.199075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:03.199075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:03.199075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:03.199985 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:03.199985 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:03.199985 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:03.199985 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:03.199985 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:03.199985 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:03.199985 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:03.199985 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:03.199985 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:03.199985 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:03.199985 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:03.199985 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:03.199985 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:03.199985 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:03.199985 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:03.199985 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:03.200776 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:03.200776 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:03.200776 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:03.200776 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:03.200776 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:03.200776 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:03.200776 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:03.200776 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:03.200776 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:03.200776 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:03.200776 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 07 15:15:03.200776 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:03.200776 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 148/299] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:15:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:03.208823 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957b6f2-af3c-4397-a495-4f41750c8d3e None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:03.214010 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293793c-6610-4c67-ae9e-b12125962125 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:03.214523 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293793c-6610-4c67-ae9e-b12125962125 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:03.214997 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293793c-6610-4c67-ae9e-b12125962125 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:03.216816 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 149/300] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:03.218914 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293793c-6610-4c67-ae9e-b12125962125 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:03.301716 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:03.323655 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957b6f2-af3c-4397-a495-4f41750c8d3e None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:03.326117 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293793c-6610-4c67-ae9e-b12125962125 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:03.359924 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca27ca2-8749-45d4-b8fd-07a6e2644a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:03.361139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca27ca2-8749-45d4-b8fd-07a6e2644a58 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:03.361423 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca27ca2-8749-45d4-b8fd-07a6e2644a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:03.361521 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de27cc-3ae0-41fa-a5dc-1af2db078ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:03.361712 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca27ca2-8749-45d4-b8fd-07a6e2644a58 None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:03.362209 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de27cc-3ae0-41fa-a5dc-1af2db078ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:03.362435 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca27ca2-8749-45d4-b8fd-07a6e2644a58 None admin] RBAC: Adding query filter params (name=member) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:03.362538 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de27cc-3ae0-41fa-a5dc-1af2db078ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:03.362739 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca27ca2-8749-45d4-b8fd-07a6e2644a58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:03.362852 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de27cc-3ae0-41fa-a5dc-1af2db078ca3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:03.364162 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca27ca2-8749-45d4-b8fd-07a6e2644a58 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:03.364809 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:03.376802 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93de27cc-3ae0-41fa-a5dc-1af2db078ca3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:03.376933 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 152/301] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Jul 7 15:15:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:03.376933 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26169c11-25ec-411f-b8ed-18c09a3b2b9b None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:03.376933 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26169c11-25ec-411f-b8ed-18c09a3b2b9b None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:03.376933 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26169c11-25ec-411f-b8ed-18c09a3b2b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:03.384387 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-93de27cc-3ae0-41fa-a5dc-1af2db078ca3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 07 15:15:03.384387 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:03.384387 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:03.384387 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:03.384387 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:03.384387 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:03.384387 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:03.384387 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:03.384387 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:03.384387 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:03.384387 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:03.384387 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:03.384387 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:03.384387 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:03.384387 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:03.384387 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:03.385315 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:03.385315 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:03.385315 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:03.385315 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:03.385315 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:03.385315 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:03.385315 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:03.385315 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:03.385315 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:03.385315 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:03.385315 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:03.385315 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:03.385315 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:03.385315 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:03.385315 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:03.385315 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:03.386198 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:03.386198 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:03.386198 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:03.386198 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:03.386198 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:03.386198 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:03.386198 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:03.386198 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:03.386198 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:03.386198 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:03.386198 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 07 15:15:03.386198 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:03.386198 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 150/302] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:15:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:15:03.386848 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de27cc-3ae0-41fa-a5dc-1af2db078ca3 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:03.399009 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-26169c11-25ec-411f-b8ed-18c09a3b2b9b None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:03.400246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-26169c11-25ec-411f-b8ed-18c09a3b2b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:03.475455 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 153/303] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:03.476620 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26169c11-25ec-411f-b8ed-18c09a3b2b9b None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:03.491978 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de27cc-3ae0-41fa-a5dc-1af2db078ca3 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:03.532041 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd9fa98-2805-45db-9224-28a3b180c293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:03.532939 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9fa98-2805-45db-9224-28a3b180c293 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:03.533229 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9fa98-2805-45db-9224-28a3b180c293 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:03.533800 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9fa98-2805-45db-9224-28a3b180c293 None admin] PATH_INFO: `/v3/users/828d69aa0f214660b3fd94c18659909d` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:03.535032 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd9fa98-2805-45db-9224-28a3b180c293 None admin] RBAC: Authorizing `identity:get_user(user_id=828d69aa0f214660b3fd94c18659909d)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:03.536514 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd9fa98-2805-45db-9224-28a3b180c293 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:03.538325 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 151/304] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:15:03 2022] GET /identity/v3/users/828d69aa0f214660b3fd94c18659909d => generated 293 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:03.538904 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd9fa98-2805-45db-9224-28a3b180c293 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:03.565217 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:03.595788 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26169c11-25ec-411f-b8ed-18c09a3b2b9b None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:03.606146 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:03.631646 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a1501-5001-4db1-bb13-898de34fab65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:03.632239 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a1501-5001-4db1-bb13-898de34fab65 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:03.632517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a1501-5001-4db1-bb13-898de34fab65 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:03.632785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a1501-5001-4db1-bb13-898de34fab65 None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:03.633532 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67a1501-5001-4db1-bb13-898de34fab65 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:03.633834 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67a1501-5001-4db1-bb13-898de34fab65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:03.635211 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67a1501-5001-4db1-bb13-898de34fab65 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:03.643993 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 154/305] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Jul 7 15:15:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:03.644670 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a1501-5001-4db1-bb13-898de34fab65 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:03.653139 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd9fa98-2805-45db-9224-28a3b180c293 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:03.690395 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121fa20f-f555-49ba-97aa-239ad7a6fe59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:03.691181 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fa20f-f555-49ba-97aa-239ad7a6fe59 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:03.691459 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fa20f-f555-49ba-97aa-239ad7a6fe59 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:03.691759 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fa20f-f555-49ba-97aa-239ad7a6fe59 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:03.692919 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121fa20f-f555-49ba-97aa-239ad7a6fe59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:03.694411 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121fa20f-f555-49ba-97aa-239ad7a6fe59 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:03.695956 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 152/306] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:03.701166 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121fa20f-f555-49ba-97aa-239ad7a6fe59 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:03.753448 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a1501-5001-4db1-bb13-898de34fab65 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:03.784441 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:03.790721 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05937644-149c-4bfd-a223-57c65f2c172f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:03.791575 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05937644-149c-4bfd-a223-57c65f2c172f None admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:03.791877 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05937644-149c-4bfd-a223-57c65f2c172f None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:03.792152 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05937644-149c-4bfd-a223-57c65f2c172f None admin] PATH_INFO: `/v3/system/users/828d69aa0f214660b3fd94c18659909d/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:03.793677 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05937644-149c-4bfd-a223-57c65f2c172f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=828d69aa0f214660b3fd94c18659909d, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:03.795046 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05937644-149c-4bfd-a223-57c65f2c172f None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:03.811799 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121fa20f-f555-49ba-97aa-239ad7a6fe59 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:03.839769 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:03.845438 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e358c-e97f-43c5-a82a-232fd79db857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:03.846302 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e358c-e97f-43c5-a82a-232fd79db857 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:03.846581 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e358c-e97f-43c5-a82a-232fd79db857 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:03.846853 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e358c-e97f-43c5-a82a-232fd79db857 None admin] PATH_INFO: `/v3/users/3a05564224f144e7a89d09d11511400a` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:03.860012 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52e358c-e97f-43c5-a82a-232fd79db857 None admin] RBAC: Authorizing `identity:get_user(user_id=3a05564224f144e7a89d09d11511400a)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:03.862003 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52e358c-e97f-43c5-a82a-232fd79db857 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:03.863663 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 153/307] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:15:03 2022] GET /identity/v3/users/3a05564224f144e7a89d09d11511400a => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:03.864399 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e358c-e97f-43c5-a82a-232fd79db857 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:03.902466 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 155/308] 10.222.0.25 () {58 vars in 1334 bytes} [Thu Jul 7 15:15:03 2022] PUT /identity/v3/system/users/828d69aa0f214660b3fd94c18659909d/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:15:03.904126 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05937644-149c-4bfd-a223-57c65f2c172f None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:03.984448 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e358c-e97f-43c5-a82a-232fd79db857 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:04.024933 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0983af-ae5c-4b7a-b6ab-d2158c193819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:04.026009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0983af-ae5c-4b7a-b6ab-d2158c193819 None admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:04.026500 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0983af-ae5c-4b7a-b6ab-d2158c193819 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:04.026998 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0983af-ae5c-4b7a-b6ab-d2158c193819 None admin] PATH_INFO: `/v3/groups` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:04.028020 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0983af-ae5c-4b7a-b6ab-d2158c193819 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:04.029715 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0983af-ae5c-4b7a-b6ab-d2158c193819 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:04.045750 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05937644-149c-4bfd-a223-57c65f2c172f None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:04.084510 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213aade7-9f98-400c-b362-a5f1af5d7656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:04.085167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213aade7-9f98-400c-b362-a5f1af5d7656 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:04.085448 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213aade7-9f98-400c-b362-a5f1af5d7656 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:04.085721 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213aade7-9f98-400c-b362-a5f1af5d7656 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:04.086564 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213aade7-9f98-400c-b362-a5f1af5d7656 None admin] RBAC: Adding query filter params (role.id=4fe04bd6b66542379945fab86d689ae6, user.id=3a05564224f144e7a89d09d11511400a, scope.system=all) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:04.086876 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213aade7-9f98-400c-b362-a5f1af5d7656 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:04.088415 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213aade7-9f98-400c-b362-a5f1af5d7656 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:04.093498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 156/309] 10.222.0.25 () {56 vars in 1308 bytes} [Thu Jul 7 15:15:03 2022] GET /identity/v3/role_assignments?user.id=3a05564224f144e7a89d09d11511400a&scope.system=all&role.id=4fe04bd6b66542379945fab86d689ae6 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:04.095524 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ae42c-aeae-4746-967d-92cc00f6dfc1 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:04.095717 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ae42c-aeae-4746-967d-92cc00f6dfc1 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:04.097282 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ae42c-aeae-4746-967d-92cc00f6dfc1 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:04.097282 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 157/310] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:04.103117 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aea825-0c27-47ca-bb49-53d4cbe671c5 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:04.103453 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aea825-0c27-47ca-bb49-53d4cbe671c5 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:04.103782 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aea825-0c27-47ca-bb49-53d4cbe671c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:04.107236 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 154/311] 10.222.0.25 () {60 vars in 1134 bytes} [Thu Jul 7 15:15:03 2022] POST /identity/v3/groups => generated 229 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:15:04.128296 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-01aea825-0c27-47ca-bb49-53d4cbe671c5 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:04.130045 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-01aea825-0c27-47ca-bb49-53d4cbe671c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:04.202825 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:04.215558 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 158/312] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 15:15:04.240033 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0983af-ae5c-4b7a-b6ab-d2158c193819 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:04.295885 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:04.354751 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0983af-ae5c-4b7a-b6ab-d2158c193819 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:04.389269 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5dd9b-accd-479d-b97c-36b83e8cb51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:04.390059 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5dd9b-accd-479d-b97c-36b83e8cb51a None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:04.390341 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5dd9b-accd-479d-b97c-36b83e8cb51a None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:04.390614 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5dd9b-accd-479d-b97c-36b83e8cb51a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:04.401667 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b5dd9b-accd-479d-b97c-36b83e8cb51a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:04.409541 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-53b5dd9b-accd-479d-b97c-36b83e8cb51a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 07 15:15:04.409541 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:04.409541 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:04.409541 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:04.409541 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:04.409541 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:04.409541 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:04.409541 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:04.409541 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:04.409541 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:04.409541 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:04.409541 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:04.409541 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:04.409541 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:04.409541 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:04.409541 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:04.410616 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:04.410616 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:04.410616 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:04.410616 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:04.410616 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:04.410616 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:04.410616 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:04.410616 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:04.410616 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:04.410616 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:04.410616 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:04.410616 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:04.410616 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:04.410616 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:04.410616 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:04.410616 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:04.411492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:04.411492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:04.411492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:04.411492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:04.411492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:04.411492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:04.411492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:04.411492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:04.411492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:04.411492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:04.411492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 07 15:15:04.411492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:04.411492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 155/313] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:15:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:04.416605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aea825-0c27-47ca-bb49-53d4cbe671c5 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:04.520441 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aea825-0c27-47ca-bb49-53d4cbe671c5 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:04.555284 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacfe1e1-b810-40dd-83ee-8b2c76cbfaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:04.555975 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacfe1e1-b810-40dd-83ee-8b2c76cbfaa5 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:04.556255 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacfe1e1-b810-40dd-83ee-8b2c76cbfaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:04.556526 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacfe1e1-b810-40dd-83ee-8b2c76cbfaa5 None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:04.557276 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacfe1e1-b810-40dd-83ee-8b2c76cbfaa5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:04.557636 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacfe1e1-b810-40dd-83ee-8b2c76cbfaa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:04.559084 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacfe1e1-b810-40dd-83ee-8b2c76cbfaa5 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:04.568323 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 159/314] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Jul 7 15:15:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:04.573967 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5dd9b-accd-479d-b97c-36b83e8cb51a None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:04.681178 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5dd9b-accd-479d-b97c-36b83e8cb51a None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:04.691351 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:04.715989 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e451cd1-00a1-4617-83e8-80fb1b5b6e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:04.716859 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e451cd1-00a1-4617-83e8-80fb1b5b6e6c None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:04.717136 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e451cd1-00a1-4617-83e8-80fb1b5b6e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:04.717407 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e451cd1-00a1-4617-83e8-80fb1b5b6e6c None admin] PATH_INFO: `/v3/users/ab5f8b8c8d574748aa17a300aaf6e2db` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:04.718595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e451cd1-00a1-4617-83e8-80fb1b5b6e6c None admin] RBAC: Authorizing `identity:get_user(user_id=ab5f8b8c8d574748aa17a300aaf6e2db)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:04.720090 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e451cd1-00a1-4617-83e8-80fb1b5b6e6c None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:04.721997 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 156/315] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:15:04 2022] GET /identity/v3/users/ab5f8b8c8d574748aa17a300aaf6e2db => generated 297 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:04.727646 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacfe1e1-b810-40dd-83ee-8b2c76cbfaa5 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:04.783021 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:04.831311 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacfe1e1-b810-40dd-83ee-8b2c76cbfaa5 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:04.866548 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6039f-590e-4cca-b125-77a3adf4293c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:04.867244 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6039f-590e-4cca-b125-77a3adf4293c None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:04.867529 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6039f-590e-4cca-b125-77a3adf4293c None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:04.867866 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6039f-590e-4cca-b125-77a3adf4293c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:04.937213 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6039f-590e-4cca-b125-77a3adf4293c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:04.938722 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6039f-590e-4cca-b125-77a3adf4293c None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:05.012927 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:05.051776 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 160/316] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:15:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 15:15:05.054014 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e451cd1-00a1-4617-83e8-80fb1b5b6e6c None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:05.159648 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e451cd1-00a1-4617-83e8-80fb1b5b6e6c None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:05.198326 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049ad482-9310-486f-a1d7-8f4bc376dc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:05.199499 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ad482-9310-486f-a1d7-8f4bc376dc1d None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:05.200115 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ad482-9310-486f-a1d7-8f4bc376dc1d None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:05.200590 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ad482-9310-486f-a1d7-8f4bc376dc1d None admin] PATH_INFO: `/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:05.201982 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049ad482-9310-486f-a1d7-8f4bc376dc1d None admin] RBAC: Authorizing `identity:get_project(project_id=a6fde692f8db4d7eb4858942b0b6e5bf)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:05.203637 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049ad482-9310-486f-a1d7-8f4bc376dc1d None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:05.206397 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 157/317] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:15:05 2022] GET /identity/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:05.211978 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6039f-590e-4cca-b125-77a3adf4293c None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:05.317149 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6039f-590e-4cca-b125-77a3adf4293c None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:05.353012 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca4f6f-5657-472a-ace6-12a48a0126f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:05.353671 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca4f6f-5657-472a-ace6-12a48a0126f5 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:05.353953 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca4f6f-5657-472a-ace6-12a48a0126f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:05.354228 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca4f6f-5657-472a-ace6-12a48a0126f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:05.355065 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ca4f6f-5657-472a-ace6-12a48a0126f5 None admin] RBAC: Adding query filter params (role.id=4fe04bd6b66542379945fab86d689ae6, scope.project.id=a6fde692f8db4d7eb4858942b0b6e5bf, user.id=ab5f8b8c8d574748aa17a300aaf6e2db) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:05.355375 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ca4f6f-5657-472a-ace6-12a48a0126f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:05.356848 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ca4f6f-5657-472a-ace6-12a48a0126f5 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:05.363275 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 161/318] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:15:05 2022] GET /identity/v3/role_assignments?user.id=ab5f8b8c8d574748aa17a300aaf6e2db&scope.project.id=a6fde692f8db4d7eb4858942b0b6e5bf&role.id=4fe04bd6b66542379945fab86d689ae6 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:06.011361 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd63408-55e1-46c7-b7fa-fe72e6f8b5ac None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:06.011759 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd63408-55e1-46c7-b7fa-fe72e6f8b5ac None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:06.011955 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd63408-55e1-46c7-b7fa-fe72e6f8b5ac None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:06.013351 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 158/319] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:06.018321 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8720c-1c6b-4132-acd4-eb6045753953 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:06.018745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8720c-1c6b-4132-acd4-eb6045753953 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:06.018899 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8720c-1c6b-4132-acd4-eb6045753953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:06.035243 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-bfb8720c-1c6b-4132-acd4-eb6045753953 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:06.036454 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-bfb8720c-1c6b-4132-acd4-eb6045753953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:06.047673 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:06.104496 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 162/320] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:06.128294 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd63408-55e1-46c7-b7fa-fe72e6f8b5ac None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:06.175988 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:06.196748 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197bc83-433b-452a-b91f-ede88652216a None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:06.197029 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197bc83-433b-452a-b91f-ede88652216a None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:06.197309 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197bc83-433b-452a-b91f-ede88652216a None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:06.198713 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 163/321] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:15:06.203755 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03a17c-5b61-486c-a6d1-9ddc87a51cd4 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:06.204043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03a17c-5b61-486c-a6d1-9ddc87a51cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:06.204346 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03a17c-5b61-486c-a6d1-9ddc87a51cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:06.220620 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-9a03a17c-5b61-486c-a6d1-9ddc87a51cd4 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:06.221712 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-9a03a17c-5b61-486c-a6d1-9ddc87a51cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:06.234545 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd63408-55e1-46c7-b7fa-fe72e6f8b5ac None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:06.271042 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851b97fd-79da-4110-9643-e7a53552eace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:06.272088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b97fd-79da-4110-9643-e7a53552eace None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:06.272540 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b97fd-79da-4110-9643-e7a53552eace None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:06.272956 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b97fd-79da-4110-9643-e7a53552eace None admin] PATH_INFO: `/v3/roles/member` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:06.283263 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851b97fd-79da-4110-9643-e7a53552eace None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:06.288963 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 164/322] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 15:15:06.291272 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-851b97fd-79da-4110-9643-e7a53552eace None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 07 15:15:06.291272 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:06.291272 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:06.291272 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:06.291272 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:06.291272 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:06.291272 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:06.291272 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:06.291272 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:06.291272 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:06.291272 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:06.291272 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:06.291272 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:06.291272 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:06.291272 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:06.291272 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:06.292374 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:06.292374 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:06.292374 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:06.292374 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:06.292374 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:06.292374 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:06.292374 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:06.292374 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:06.292374 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:06.292374 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:06.292374 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:06.292374 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:06.292374 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:06.292374 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:06.292374 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:06.292374 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:06.293381 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:06.293381 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:06.293381 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:06.293381 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:06.293381 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:06.293381 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:06.293381 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:06.293381 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:06.293381 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:06.293381 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:06.293381 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 07 15:15:06.293381 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:06.295353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 159/323] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:15:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:06.303596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a03a17c-5b61-486c-a6d1-9ddc87a51cd4 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:06.308897 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c7ca8f-cec4-48b2-bbc3-c96165fb6704 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:06.309190 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c7ca8f-cec4-48b2-bbc3-c96165fb6704 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:06.309468 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c7ca8f-cec4-48b2-bbc3-c96165fb6704 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:06.310976 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 160/324] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:06.312211 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c7ca8f-cec4-48b2-bbc3-c96165fb6704 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:06.378425 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:06.403320 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a03a17c-5b61-486c-a6d1-9ddc87a51cd4 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:06.419415 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c7ca8f-cec4-48b2-bbc3-c96165fb6704 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:06.421793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:06.440520 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c9b06e-5749-4060-8437-b2a8c8407b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:06.441186 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9b06e-5749-4060-8437-b2a8c8407b4f None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:06.441466 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9b06e-5749-4060-8437-b2a8c8407b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:06.441738 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9b06e-5749-4060-8437-b2a8c8407b4f None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:06.442482 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c9b06e-5749-4060-8437-b2a8c8407b4f None admin] RBAC: Adding query filter params (name=member) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:06.442784 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c9b06e-5749-4060-8437-b2a8c8407b4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:06.444191 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c9b06e-5749-4060-8437-b2a8c8407b4f None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:06.452173 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 165/325] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Jul 7 15:15:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:06.453321 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20750e-5762-489f-a1f3-2ded57b2cdd8 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:06.453603 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20750e-5762-489f-a1f3-2ded57b2cdd8 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:06.453876 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20750e-5762-489f-a1f3-2ded57b2cdd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:06.454662 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c66c62-3127-428a-8c97-1a2298882957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:06.455448 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c66c62-3127-428a-8c97-1a2298882957 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:06.455759 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c66c62-3127-428a-8c97-1a2298882957 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:06.456033 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c66c62-3127-428a-8c97-1a2298882957 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:06.466397 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c66c62-3127-428a-8c97-1a2298882957 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:06.469483 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-bc20750e-5762-489f-a1f3-2ded57b2cdd8 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:06.471235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-bc20750e-5762-489f-a1f3-2ded57b2cdd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:06.474873 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-62c66c62-3127-428a-8c97-1a2298882957 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 07 15:15:06.474873 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:06.474873 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:06.474873 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:06.474873 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:06.474873 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:06.474873 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:06.474873 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:06.474873 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:06.474873 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:06.474873 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:06.474873 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:06.474873 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:06.474873 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:06.474873 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:06.474873 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:06.475793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:06.475793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:06.475793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:06.475793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:06.475793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:06.475793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:06.475793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:06.475793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:06.475793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:06.475793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:06.475793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:06.475793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:06.475793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:06.475793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:06.475793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:06.475793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:06.476590 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:06.476590 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:06.476590 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:06.476590 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:06.476590 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:06.476590 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:06.476590 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:06.476590 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:06.476590 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:06.476590 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:06.476590 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 07 15:15:06.476590 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:06.476590 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 161/326] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:15:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:15:06.477871 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c66c62-3127-428a-8c97-1a2298882957 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:06.537249 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 166/327] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:06.538309 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20750e-5762-489f-a1f3-2ded57b2cdd8 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:06.578810 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c66c62-3127-428a-8c97-1a2298882957 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:06.605930 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:06.613087 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f3cbc-6752-4653-9efa-de907efe740e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:06.613943 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f3cbc-6752-4653-9efa-de907efe740e None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:06.614223 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f3cbc-6752-4653-9efa-de907efe740e None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:06.618433 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f3cbc-6752-4653-9efa-de907efe740e None admin] PATH_INFO: `/v3/users/828d69aa0f214660b3fd94c18659909d` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:06.618433 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045f3cbc-6752-4653-9efa-de907efe740e None admin] RBAC: Authorizing `identity:get_user(user_id=828d69aa0f214660b3fd94c18659909d)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:06.618433 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045f3cbc-6752-4653-9efa-de907efe740e None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:06.618705 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 162/328] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:15:06 2022] GET /identity/v3/users/828d69aa0f214660b3fd94c18659909d => generated 293 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:06.623716 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f3cbc-6752-4653-9efa-de907efe740e None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:06.639165 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20750e-5762-489f-a1f3-2ded57b2cdd8 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:06.653401 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:06.679421 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d09ee-9861-4c9f-b256-b575f3f66a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:06.680524 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d09ee-9861-4c9f-b256-b575f3f66a13 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:06.680524 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d09ee-9861-4c9f-b256-b575f3f66a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:06.680703 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d09ee-9861-4c9f-b256-b575f3f66a13 None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:06.681511 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893d09ee-9861-4c9f-b256-b575f3f66a13 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:06.681820 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893d09ee-9861-4c9f-b256-b575f3f66a13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:06.683228 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893d09ee-9861-4c9f-b256-b575f3f66a13 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:06.694193 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 167/329] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Jul 7 15:15:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:06.695003 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d09ee-9861-4c9f-b256-b575f3f66a13 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:06.732751 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f3cbc-6752-4653-9efa-de907efe740e None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:06.768168 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38b14d6-39f5-4dae-b613-42cef7989ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:06.768953 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b14d6-39f5-4dae-b613-42cef7989ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:06.769240 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b14d6-39f5-4dae-b613-42cef7989ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:06.769542 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b14d6-39f5-4dae-b613-42cef7989ea7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:06.770707 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38b14d6-39f5-4dae-b613-42cef7989ea7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:06.772187 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38b14d6-39f5-4dae-b613-42cef7989ea7 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:06.773796 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 163/330] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:06.774556 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38b14d6-39f5-4dae-b613-42cef7989ea7 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:06.794292 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d09ee-9861-4c9f-b256-b575f3f66a13 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:06.821409 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:06.826900 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785bcf9-3423-439c-bd51-9351f892fbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:06.827542 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785bcf9-3423-439c-bd51-9351f892fbfd None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:06.827849 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785bcf9-3423-439c-bd51-9351f892fbfd None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:06.828120 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785bcf9-3423-439c-bd51-9351f892fbfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:06.828945 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3785bcf9-3423-439c-bd51-9351f892fbfd None admin] RBAC: Adding query filter params (role.id=d43d0d189e254500875f5f86e6abd178, user.id=828d69aa0f214660b3fd94c18659909d, scope.system=all) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:06.829786 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3785bcf9-3423-439c-bd51-9351f892fbfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:06.831206 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3785bcf9-3423-439c-bd51-9351f892fbfd None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:06.842740 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 168/331] 10.222.0.25 () {56 vars in 1308 bytes} [Thu Jul 7 15:15:06 2022] GET /identity/v3/role_assignments?user.id=828d69aa0f214660b3fd94c18659909d&scope.system=all&role.id=d43d0d189e254500875f5f86e6abd178 => generated 510 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:06.843318 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785bcf9-3423-439c-bd51-9351f892fbfd None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:06.879263 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:06.883425 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38b14d6-39f5-4dae-b613-42cef7989ea7 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:06.923428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e42e96-2d2d-4a68-9630-102abbbda406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:06.924344 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e42e96-2d2d-4a68-9630-102abbbda406 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:06.924627 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e42e96-2d2d-4a68-9630-102abbbda406 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:06.924901 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e42e96-2d2d-4a68-9630-102abbbda406 None admin] PATH_INFO: `/v3/users/3a05564224f144e7a89d09d11511400a` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:06.926340 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e42e96-2d2d-4a68-9630-102abbbda406 None admin] RBAC: Authorizing `identity:get_user(user_id=3a05564224f144e7a89d09d11511400a)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:06.927840 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e42e96-2d2d-4a68-9630-102abbbda406 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:06.929815 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 164/332] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:15:06 2022] GET /identity/v3/users/3a05564224f144e7a89d09d11511400a => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:06.935339 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e42e96-2d2d-4a68-9630-102abbbda406 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:06.969275 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785bcf9-3423-439c-bd51-9351f892fbfd None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:07.023172 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d791c7f-24e0-4a43-9871-076b1d7a2460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:07.024355 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d791c7f-24e0-4a43-9871-076b1d7a2460 None admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:07.024642 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d791c7f-24e0-4a43-9871-076b1d7a2460 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:07.024922 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d791c7f-24e0-4a43-9871-076b1d7a2460 None admin] PATH_INFO: `/v3/groups` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:07.025622 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d791c7f-24e0-4a43-9871-076b1d7a2460 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:07.027073 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d791c7f-24e0-4a43-9871-076b1d7a2460 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:07.064848 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e42e96-2d2d-4a68-9630-102abbbda406 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:07.099197 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c801b8-f980-4341-b038-4922b91d3992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:07.100086 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c801b8-f980-4341-b038-4922b91d3992 None admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:07.100365 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c801b8-f980-4341-b038-4922b91d3992 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:07.100639 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c801b8-f980-4341-b038-4922b91d3992 None admin] PATH_INFO: `/v3/system/users/3a05564224f144e7a89d09d11511400a/roles/4fe04bd6b66542379945fab86d689ae6` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:07.102098 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c801b8-f980-4341-b038-4922b91d3992 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3a05564224f144e7a89d09d11511400a, role_id=4fe04bd6b66542379945fab86d689ae6)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:07.103457 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c801b8-f980-4341-b038-4922b91d3992 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:07.122255 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 169/333] 10.222.0.25 () {60 vars in 1134 bytes} [Thu Jul 7 15:15:06 2022] POST /identity/v3/groups => generated 226 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:15:07.168988 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 165/334] 10.222.0.25 () {58 vars in 1334 bytes} [Thu Jul 7 15:15:06 2022] PUT /identity/v3/system/users/3a05564224f144e7a89d09d11511400a/roles/4fe04bd6b66542379945fab86d689ae6 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:15:09.457010 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4e227-5555-4c33-9cc8-4d1721f269fa None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:09.457528 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4e227-5555-4c33-9cc8-4d1721f269fa None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:09.457836 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4e227-5555-4c33-9cc8-4d1721f269fa None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:09.459545 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 170/335] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:09.469514 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2061aa0-eca7-4dca-ba21-52317072f83a None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:09.470943 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2061aa0-eca7-4dca-ba21-52317072f83a None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:09.471421 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2061aa0-eca7-4dca-ba21-52317072f83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:09.492893 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-f2061aa0-eca7-4dca-ba21-52317072f83a None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:09.494428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-f2061aa0-eca7-4dca-ba21-52317072f83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:09.526545 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:09.580216 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 166/336] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:09.641115 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c4e227-5555-4c33-9cc8-4d1721f269fa None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:09.731444 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:09.810814 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217082e-f715-4850-b93e-b4a28b29c23f None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:09.811184 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217082e-f715-4850-b93e-b4a28b29c23f None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:09.811377 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217082e-f715-4850-b93e-b4a28b29c23f None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:09.816595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 167/337] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:09.821308 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c4e227-5555-4c33-9cc8-4d1721f269fa None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:09.821907 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed754bea-36e8-406a-8d76-2ab9b56dfada None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:09.822193 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed754bea-36e8-406a-8d76-2ab9b56dfada None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:09.822470 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed754bea-36e8-406a-8d76-2ab9b56dfada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:09.859005 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-ed754bea-36e8-406a-8d76-2ab9b56dfada None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:09.860328 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-ed754bea-36e8-406a-8d76-2ab9b56dfada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:09.888316 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c112741-9905-4182-9b65-2750080c9b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:09.889134 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c112741-9905-4182-9b65-2750080c9b7c None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:09.889416 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c112741-9905-4182-9b65-2750080c9b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:09.889689 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c112741-9905-4182-9b65-2750080c9b7c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:09.905410 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c112741-9905-4182-9b65-2750080c9b7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:09.919179 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-3c112741-9905-4182-9b65-2750080c9b7c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 07 15:15:09.919179 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:09.919179 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:09.919179 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:09.919179 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:09.919179 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:09.919179 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:09.919179 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:09.919179 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:09.919179 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:09.919179 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:09.919179 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:09.919179 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:09.919179 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:09.919179 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:09.919179 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:09.920430 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:09.920430 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:09.920430 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:09.920430 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:09.920430 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:09.920430 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:09.920430 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:09.920430 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:09.920430 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:09.920430 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:09.920430 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:09.920430 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:09.920430 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:09.920430 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:09.920430 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:09.920430 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:09.921290 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:09.921290 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:09.921290 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:09.921290 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:09.921290 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:09.921290 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:09.921290 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:09.921290 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:09.921290 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:09.921290 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:09.921290 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 07 15:15:09.921290 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:09.921290 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 171/338] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:15:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:09.922449 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b79b9b-b5d0-4d51-b5f5-ddfaa34d4d18 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:09.923193 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b79b9b-b5d0-4d51-b5f5-ddfaa34d4d18 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:09.923193 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b79b9b-b5d0-4d51-b5f5-ddfaa34d4d18 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:09.924588 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 172/339] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:09.929826 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427f037-e823-41a4-8196-445d7250e3c9 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:09.930114 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427f037-e823-41a4-8196-445d7250e3c9 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:09.930514 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427f037-e823-41a4-8196-445d7250e3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:09.965468 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-e427f037-e823-41a4-8196-445d7250e3c9 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:09.966703 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-e427f037-e823-41a4-8196-445d7250e3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:10.007981 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 168/340] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:10.009170 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed754bea-36e8-406a-8d76-2ab9b56dfada None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:10.083999 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 173/341] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:10.085233 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e427f037-e823-41a4-8196-445d7250e3c9 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:10.169802 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:10.190206 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:10.201849 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed754bea-36e8-406a-8d76-2ab9b56dfada None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:10.242303 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dbfbfb-3803-4364-8d60-82c226e03b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:10.243318 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbfbfb-3803-4364-8d60-82c226e03b66 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:10.243858 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbfbfb-3803-4364-8d60-82c226e03b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:10.244334 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbfbfb-3803-4364-8d60-82c226e03b66 None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:10.245289 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dbfbfb-3803-4364-8d60-82c226e03b66 None admin] RBAC: Adding query filter params (name=member) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:10.247098 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dbfbfb-3803-4364-8d60-82c226e03b66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:10.249548 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dbfbfb-3803-4364-8d60-82c226e03b66 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:10.267877 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 169/342] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Jul 7 15:15:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:10.268531 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dbfbfb-3803-4364-8d60-82c226e03b66 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:10.289517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e427f037-e823-41a4-8196-445d7250e3c9 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:10.325987 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d9d4da-8dc2-4d0a-8e71-4630923fabdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:10.327702 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d9d4da-8dc2-4d0a-8e71-4630923fabdc None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:10.328244 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d9d4da-8dc2-4d0a-8e71-4630923fabdc None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:10.328714 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d9d4da-8dc2-4d0a-8e71-4630923fabdc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:10.340450 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d9d4da-8dc2-4d0a-8e71-4630923fabdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:10.349193 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-44d9d4da-8dc2-4d0a-8e71-4630923fabdc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 15:15:10.349193 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:10.349193 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:10.349193 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:10.349193 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:10.349193 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:10.349193 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:10.349193 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:10.349193 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:10.349193 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:10.349193 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:10.349193 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:10.349193 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:10.349193 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:10.349193 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:10.349193 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:10.350101 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:10.350101 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:10.350101 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:10.350101 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:10.350101 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:10.350101 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:10.350101 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:10.350101 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:10.350101 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:10.350101 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:10.350101 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:10.350101 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:10.350101 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:10.350101 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:10.350101 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:10.350101 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:10.350902 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:10.350902 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:10.350902 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:10.350902 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:10.350902 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:10.350902 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:10.350902 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:10.350902 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:10.350902 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:10.350902 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:10.350902 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 15:15:10.350902 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:10.353617 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 174/343] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Jul 7 15:15:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:10.355022 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005cee2-ec1d-4345-b3d8-a97e4d87c495 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:10.355507 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005cee2-ec1d-4345-b3d8-a97e4d87c495 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:10.356001 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005cee2-ec1d-4345-b3d8-a97e4d87c495 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:10.361727 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 175/344] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:10.362730 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005cee2-ec1d-4345-b3d8-a97e4d87c495 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:10.397922 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dbfbfb-3803-4364-8d60-82c226e03b66 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:10.440111 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf06f58c-a3c3-4e50-bd0e-1a5a29028f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:10.441428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06f58c-a3c3-4e50-bd0e-1a5a29028f44 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:10.441428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06f58c-a3c3-4e50-bd0e-1a5a29028f44 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:10.441428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06f58c-a3c3-4e50-bd0e-1a5a29028f44 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:10.452018 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf06f58c-a3c3-4e50-bd0e-1a5a29028f44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:10.466029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-cf06f58c-a3c3-4e50-bd0e-1a5a29028f44 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 07 15:15:10.466029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:10.466029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:10.466029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:10.466029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:10.466029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:10.466029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:10.466029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:10.466029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:10.466029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:10.466029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:10.466029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:10.466029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:10.466029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:10.466029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:10.466029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:10.466999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:10.466999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:10.466999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:10.466999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:10.466999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:10.466999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:10.466999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:10.466999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:10.466999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:10.466999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:10.466999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:10.466999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:10.466999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:10.466999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:10.466999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:10.466999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:10.469489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:10.469489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:10.469489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:10.469489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:10.469489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:10.469489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:10.469489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:10.469489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:10.469489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:10.469489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:10.469489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 07 15:15:10.469489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:10.469489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 170/345] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:15:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:10.469489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b6378-729e-401a-9fde-97bb43bc94f1 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:10.470672 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b6378-729e-401a-9fde-97bb43bc94f1 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:10.470672 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b6378-729e-401a-9fde-97bb43bc94f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:10.478615 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:10.483882 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005cee2-ec1d-4345-b3d8-a97e4d87c495 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:10.488700 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-f92b6378-729e-401a-9fde-97bb43bc94f1 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:10.490243 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-f92b6378-729e-401a-9fde-97bb43bc94f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:10.517671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:10.526918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567adf2-dd7b-4ca7-b6e6-83b87a854144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:10.528229 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567adf2-dd7b-4ca7-b6e6-83b87a854144 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:10.528600 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567adf2-dd7b-4ca7-b6e6-83b87a854144 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:10.528947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567adf2-dd7b-4ca7-b6e6-83b87a854144 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:10.595527 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 171/346] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:10.596459 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b6378-729e-401a-9fde-97bb43bc94f1 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:10.653076 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5567adf2-dd7b-4ca7-b6e6-83b87a854144 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:10.654286 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5567adf2-dd7b-4ca7-b6e6-83b87a854144 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:10.734723 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b6378-729e-401a-9fde-97bb43bc94f1 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:10.775011 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0026e21-5f81-4b0a-89fc-8d2438000d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:10.775713 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0026e21-5f81-4b0a-89fc-8d2438000d3c None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:10.775992 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0026e21-5f81-4b0a-89fc-8d2438000d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:10.776259 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0026e21-5f81-4b0a-89fc-8d2438000d3c None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:10.777074 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0026e21-5f81-4b0a-89fc-8d2438000d3c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:10.777372 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0026e21-5f81-4b0a-89fc-8d2438000d3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:10.778766 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0026e21-5f81-4b0a-89fc-8d2438000d3c None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:10.785490 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:10.788163 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 172/347] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:15:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:10.788996 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0026e21-5f81-4b0a-89fc-8d2438000d3c None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:10.795839 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 176/348] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:15:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 15:15:10.798829 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567adf2-dd7b-4ca7-b6e6-83b87a854144 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:10.803853 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:10.917089 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0026e21-5f81-4b0a-89fc-8d2438000d3c None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:10.937108 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567adf2-dd7b-4ca7-b6e6-83b87a854144 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:10.956598 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fd48a8-5e72-4231-91d6-f0ecc555898e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:10.957302 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd48a8-5e72-4231-91d6-f0ecc555898e None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:10.957658 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd48a8-5e72-4231-91d6-f0ecc555898e None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:10.958000 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd48a8-5e72-4231-91d6-f0ecc555898e None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:10.958918 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fd48a8-5e72-4231-91d6-f0ecc555898e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:10.959274 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fd48a8-5e72-4231-91d6-f0ecc555898e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:10.961955 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fd48a8-5e72-4231-91d6-f0ecc555898e None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:10.972252 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 173/349] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Jul 7 15:15:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:10.973417 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fd48a8-5e72-4231-91d6-f0ecc555898e None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:10.983939 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1049c6e-5cbc-4606-bda4-ba5ae5fe94ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:10.984993 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1049c6e-5cbc-4606-bda4-ba5ae5fe94ac None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:10.986008 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1049c6e-5cbc-4606-bda4-ba5ae5fe94ac None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:10.986505 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1049c6e-5cbc-4606-bda4-ba5ae5fe94ac None admin] PATH_INFO: `/v3/roles/member` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:11.007826 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1049c6e-5cbc-4606-bda4-ba5ae5fe94ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:11.015541 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-d1049c6e-5cbc-4606-bda4-ba5ae5fe94ac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 07 15:15:11.015541 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:11.015541 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:11.015541 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:11.015541 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:11.015541 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:11.015541 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:11.015541 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:11.015541 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:11.015541 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:11.015541 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:11.015541 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:11.015541 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:11.015541 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:11.015541 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:11.015541 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:11.016492 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:11.016492 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:11.016492 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:11.016492 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:11.016492 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:11.016492 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:11.016492 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:11.016492 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:11.016492 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:11.016492 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:11.016492 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:11.016492 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:11.016492 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:11.016492 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:11.016492 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:11.016492 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:11.017287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:11.017287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:11.017287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:11.017287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:11.017287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:11.017287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:11.017287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:11.017287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:11.017287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:11.017287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:11.017287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 07 15:15:11.017287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:11.025064 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 177/350] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:15:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:11.025064 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1049c6e-5cbc-4606-bda4-ba5ae5fe94ac None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:11.080133 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:11.093900 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:11.120601 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fd48a8-5e72-4231-91d6-f0ecc555898e None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:11.153709 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1049c6e-5cbc-4606-bda4-ba5ae5fe94ac None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:11.170892 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c64be-b946-4b0e-a7e3-2e08e14e9f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:11.171945 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c64be-b946-4b0e-a7e3-2e08e14e9f41 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:11.172432 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c64be-b946-4b0e-a7e3-2e08e14e9f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:11.173475 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c64be-b946-4b0e-a7e3-2e08e14e9f41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:11.192478 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208df838-ff0f-4045-8087-5e8138364c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:11.193335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208df838-ff0f-4045-8087-5e8138364c68 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:11.193621 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208df838-ff0f-4045-8087-5e8138364c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:11.193890 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208df838-ff0f-4045-8087-5e8138364c68 None admin] PATH_INFO: `/v3/projects/db427256e2ec4ec084f762a615d0f187` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:11.195099 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208df838-ff0f-4045-8087-5e8138364c68 None admin] RBAC: Authorizing `identity:get_project(project_id=db427256e2ec4ec084f762a615d0f187)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:11.196576 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208df838-ff0f-4045-8087-5e8138364c68 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:11.198421 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 178/351] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:15:11 2022] GET /identity/v3/projects/db427256e2ec4ec084f762a615d0f187 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:11.199007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208df838-ff0f-4045-8087-5e8138364c68 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:11.258641 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c64be-b946-4b0e-a7e3-2e08e14e9f41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:11.260133 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c64be-b946-4b0e-a7e3-2e08e14e9f41 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:11.320538 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208df838-ff0f-4045-8087-5e8138364c68 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:11.347417 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:11.349989 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:11.364748 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856f48db-a09b-441d-b51b-a2898057d0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:11.366239 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f48db-a09b-441d-b51b-a2898057d0b2 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:11.366719 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f48db-a09b-441d-b51b-a2898057d0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:11.367318 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f48db-a09b-441d-b51b-a2898057d0b2 None admin] PATH_INFO: `/v3/users/3a05564224f144e7a89d09d11511400a` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:11.369270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856f48db-a09b-441d-b51b-a2898057d0b2 None admin] RBAC: Authorizing `identity:get_user(user_id=3a05564224f144e7a89d09d11511400a)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:11.371137 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856f48db-a09b-441d-b51b-a2898057d0b2 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:11.374220 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 179/352] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:15:11 2022] GET /identity/v3/users/3a05564224f144e7a89d09d11511400a => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:11.375369 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856f48db-a09b-441d-b51b-a2898057d0b2 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:11.387684 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 174/353] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:15:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 15:15:11.389177 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8eeb9-1406-4b1c-b4bf-67e6433e92ea None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:11.389469 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8eeb9-1406-4b1c-b4bf-67e6433e92ea None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:11.389743 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8eeb9-1406-4b1c-b4bf-67e6433e92ea None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:11.391138 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 175/354] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:11.394858 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f8eeb9-1406-4b1c-b4bf-67e6433e92ea None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:11.523445 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856f48db-a09b-441d-b51b-a2898057d0b2 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:11.540904 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f8eeb9-1406-4b1c-b4bf-67e6433e92ea None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:11.578668 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53365cbe-28eb-43f5-8cc6-44e5b251f821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:11.580503 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53365cbe-28eb-43f5-8cc6-44e5b251f821 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:11.580851 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53365cbe-28eb-43f5-8cc6-44e5b251f821 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:11.581193 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53365cbe-28eb-43f5-8cc6-44e5b251f821 None admin] PATH_INFO: `/v3/groups/dd675b91e3ef45dc8650e456e0529fd7` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:11.584233 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd417b6e-02fb-4f9c-9d2a-190763430a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:11.585110 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd417b6e-02fb-4f9c-9d2a-190763430a85 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:11.585564 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd417b6e-02fb-4f9c-9d2a-190763430a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:11.586007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd417b6e-02fb-4f9c-9d2a-190763430a85 None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:11.586960 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd417b6e-02fb-4f9c-9d2a-190763430a85 None admin] RBAC: Adding query filter params (name=member) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:11.587451 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd417b6e-02fb-4f9c-9d2a-190763430a85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:11.589928 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd417b6e-02fb-4f9c-9d2a-190763430a85 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:11.590407 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53365cbe-28eb-43f5-8cc6-44e5b251f821 None admin] RBAC: Authorizing `identity:get_group(group_id=dd675b91e3ef45dc8650e456e0529fd7)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:11.591950 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53365cbe-28eb-43f5-8cc6-44e5b251f821 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:11.593886 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 176/355] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Jul 7 15:15:11 2022] GET /identity/v3/groups/dd675b91e3ef45dc8650e456e0529fd7 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:11.596378 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53365cbe-28eb-43f5-8cc6-44e5b251f821 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:11.600554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 180/356] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Jul 7 15:15:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:11.601310 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd417b6e-02fb-4f9c-9d2a-190763430a85 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:11.636837 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:11.673067 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:11.731145 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd417b6e-02fb-4f9c-9d2a-190763430a85 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:11.760292 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53365cbe-28eb-43f5-8cc6-44e5b251f821 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:11.799209 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ec12e-4087-4acf-afe4-da5621d8d8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:11.800506 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ec12e-4087-4acf-afe4-da5621d8d8ff None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:11.800989 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ec12e-4087-4acf-afe4-da5621d8d8ff None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:11.801461 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ec12e-4087-4acf-afe4-da5621d8d8ff None admin] PATH_INFO: `/v3/projects/db427256e2ec4ec084f762a615d0f187` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:11.803064 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958ec12e-4087-4acf-afe4-da5621d8d8ff None admin] RBAC: Authorizing `identity:get_project(project_id=db427256e2ec4ec084f762a615d0f187)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:11.804982 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958ec12e-4087-4acf-afe4-da5621d8d8ff None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:11.807178 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 181/357] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:15:11 2022] GET /identity/v3/projects/db427256e2ec4ec084f762a615d0f187 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:11.811272 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d513d92-f616-4018-a005-7a836563fb12 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:11.811926 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d513d92-f616-4018-a005-7a836563fb12 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:11.812501 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d513d92-f616-4018-a005-7a836563fb12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:11.815964 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac96b1b8-265a-48f0-b6bb-fb25a78486de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:11.816623 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96b1b8-265a-48f0-b6bb-fb25a78486de None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:11.816903 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96b1b8-265a-48f0-b6bb-fb25a78486de None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:11.817177 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96b1b8-265a-48f0-b6bb-fb25a78486de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:11.819559 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac96b1b8-265a-48f0-b6bb-fb25a78486de None admin] RBAC: Adding query filter params (role.id=4fe04bd6b66542379945fab86d689ae6, user.id=3a05564224f144e7a89d09d11511400a, scope.system=all) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:11.819889 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac96b1b8-265a-48f0-b6bb-fb25a78486de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:11.823904 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac96b1b8-265a-48f0-b6bb-fb25a78486de None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:11.836139 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 177/358] 10.222.0.25 () {56 vars in 1308 bytes} [Thu Jul 7 15:15:11 2022] GET /identity/v3/role_assignments?user.id=3a05564224f144e7a89d09d11511400a&scope.system=all&role.id=4fe04bd6b66542379945fab86d689ae6 => generated 510 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:11.836989 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac96b1b8-265a-48f0-b6bb-fb25a78486de None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:11.840122 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-0d513d92-f616-4018-a005-7a836563fb12 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:11.841963 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-0d513d92-f616-4018-a005-7a836563fb12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:11.945624 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 182/359] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:11.945624 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d513d92-f616-4018-a005-7a836563fb12 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:11.947924 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:11.972258 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac96b1b8-265a-48f0-b6bb-fb25a78486de None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:12.017574 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:12.024588 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93d3841-1ee3-44b3-a852-2d8ff6432c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:12.024588 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d3841-1ee3-44b3-a852-2d8ff6432c8b None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:12.024882 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d3841-1ee3-44b3-a852-2d8ff6432c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:12.024959 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d3841-1ee3-44b3-a852-2d8ff6432c8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:12.025782 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93d3841-1ee3-44b3-a852-2d8ff6432c8b None admin] RBAC: Adding query filter params (group.id=dd675b91e3ef45dc8650e456e0529fd7, role.id=d43d0d189e254500875f5f86e6abd178, scope.project.id=db427256e2ec4ec084f762a615d0f187) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:12.026088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93d3841-1ee3-44b3-a852-2d8ff6432c8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:12.027548 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93d3841-1ee3-44b3-a852-2d8ff6432c8b None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:12.036281 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 178/360] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Jul 7 15:15:11 2022] GET /identity/v3/role_assignments?group.id=dd675b91e3ef45dc8650e456e0529fd7&scope.project.id=db427256e2ec4ec084f762a615d0f187&role.id=d43d0d189e254500875f5f86e6abd178 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:12.037300 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93d3841-1ee3-44b3-a852-2d8ff6432c8b None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:12.098702 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d513d92-f616-4018-a005-7a836563fb12 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:12.160680 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a2091-ef65-42e1-a311-91d8517a4290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:12.161607 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a2091-ef65-42e1-a311-91d8517a4290 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:12.161884 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a2091-ef65-42e1-a311-91d8517a4290 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:12.162155 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a2091-ef65-42e1-a311-91d8517a4290 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:12.208996 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93d3841-1ee3-44b3-a852-2d8ff6432c8b None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:12.257328 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6ec5e-5454-49f2-89a7-16d4139415a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:12.258191 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6ec5e-5454-49f2-89a7-16d4139415a4 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:12.258460 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6ec5e-5454-49f2-89a7-16d4139415a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:12.258709 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6ec5e-5454-49f2-89a7-16d4139415a4 None admin] PATH_INFO: `/v3/groups/dd675b91e3ef45dc8650e456e0529fd7` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:12.259305 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615a2091-ef65-42e1-a311-91d8517a4290 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:12.261457 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd6ec5e-5454-49f2-89a7-16d4139415a4 None admin] RBAC: Authorizing `identity:get_group(group_id=dd675b91e3ef45dc8650e456e0529fd7)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:12.261583 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615a2091-ef65-42e1-a311-91d8517a4290 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:12.261883 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd6ec5e-5454-49f2-89a7-16d4139415a4 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:12.263747 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 179/361] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Jul 7 15:15:12 2022] GET /identity/v3/groups/dd675b91e3ef45dc8650e456e0529fd7 => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:12.265443 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2628deb-1216-4656-8efd-75e11254eeaa None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:12.265725 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2628deb-1216-4656-8efd-75e11254eeaa None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:12.265997 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2628deb-1216-4656-8efd-75e11254eeaa None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:12.270633 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 180/362] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:12.270633 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2628deb-1216-4656-8efd-75e11254eeaa None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:12.298321 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:12.342915 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:12.403105 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 183/363] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:15:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 15:15:12.403868 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a2091-ef65-42e1-a311-91d8517a4290 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:12.414400 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2628deb-1216-4656-8efd-75e11254eeaa None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:12.456198 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb63eda-fb31-4c4e-ae29-efa857fdc35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:12.457011 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb63eda-fb31-4c4e-ae29-efa857fdc35d None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:12.457292 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb63eda-fb31-4c4e-ae29-efa857fdc35d None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:12.457566 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb63eda-fb31-4c4e-ae29-efa857fdc35d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:12.473846 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb63eda-fb31-4c4e-ae29-efa857fdc35d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:12.483803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-fdb63eda-fb31-4c4e-ae29-efa857fdc35d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 15:15:12.483803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:12.483803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:12.483803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:12.483803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:12.483803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:12.483803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:12.483803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:12.483803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:12.483803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:12.483803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:12.483803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:12.483803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:12.483803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:12.483803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:12.483803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:12.484790 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:12.484790 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:12.484790 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:12.484790 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:12.484790 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:12.484790 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:12.484790 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:12.484790 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:12.484790 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:12.484790 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:12.484790 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:12.484790 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:12.484790 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:12.484790 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:12.484790 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:12.484790 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:12.486196 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:12.486196 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:12.486196 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:12.486196 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:12.486196 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:12.486196 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:12.486196 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:12.486196 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:12.486196 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:12.486196 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:12.486196 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 15:15:12.486196 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:12.488685 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 181/364] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Jul 7 15:15:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:12.491771 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45279f1b-512f-40b1-9a93-7910b8db0ee7 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:12.492117 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45279f1b-512f-40b1-9a93-7910b8db0ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:12.492455 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45279f1b-512f-40b1-9a93-7910b8db0ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:12.511421 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-45279f1b-512f-40b1-9a93-7910b8db0ee7 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:12.512723 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-45279f1b-512f-40b1-9a93-7910b8db0ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:12.536365 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a2091-ef65-42e1-a311-91d8517a4290 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:12.579383 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dafa978-2de2-42a8-8ce1-724ad7bd1fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:12.580148 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafa978-2de2-42a8-8ce1-724ad7bd1fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:12.580434 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafa978-2de2-42a8-8ce1-724ad7bd1fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:12.580717 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafa978-2de2-42a8-8ce1-724ad7bd1fb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:12.581914 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dafa978-2de2-42a8-8ce1-724ad7bd1fb7 None admin] RBAC: Adding query filter params (group.id=dd675b91e3ef45dc8650e456e0529fd7, role.id=c82203e4734346719cafb16230f45942, scope.project.id=db427256e2ec4ec084f762a615d0f187) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:12.582270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dafa978-2de2-42a8-8ce1-724ad7bd1fb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:12.584323 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dafa978-2de2-42a8-8ce1-724ad7bd1fb7 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:12.593371 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 184/365] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Jul 7 15:15:12 2022] GET /identity/v3/role_assignments?group.id=dd675b91e3ef45dc8650e456e0529fd7&scope.project.id=db427256e2ec4ec084f762a615d0f187&role.id=c82203e4734346719cafb16230f45942 => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:12.594988 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dafa978-2de2-42a8-8ce1-724ad7bd1fb7 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:12.602746 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 182/366] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:12.603781 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45279f1b-512f-40b1-9a93-7910b8db0ee7 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:12.618894 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:12.619391 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:12.714933 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dafa978-2de2-42a8-8ce1-724ad7bd1fb7 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:12.718626 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45279f1b-512f-40b1-9a93-7910b8db0ee7 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:12.781671 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b4775-5a38-4d3a-a644-8c5357c508ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:12.783545 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86c1665-27b7-45d2-b913-b084f7aaf16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:12.784152 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b4775-5a38-4d3a-a644-8c5357c508ce None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:12.784475 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c1665-27b7-45d2-b913-b084f7aaf16a None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:12.784888 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c1665-27b7-45d2-b913-b084f7aaf16a None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:12.785240 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b4775-5a38-4d3a-a644-8c5357c508ce None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:12.785404 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c1665-27b7-45d2-b913-b084f7aaf16a None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:12.785972 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b4775-5a38-4d3a-a644-8c5357c508ce None admin] PATH_INFO: `/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:12.786329 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86c1665-27b7-45d2-b913-b084f7aaf16a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:12.786739 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86c1665-27b7-45d2-b913-b084f7aaf16a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:12.788287 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86c1665-27b7-45d2-b913-b084f7aaf16a None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:12.789173 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b4775-5a38-4d3a-a644-8c5357c508ce None admin] RBAC: Authorizing `identity:get_project(project_id=a6fde692f8db4d7eb4858942b0b6e5bf)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:12.793115 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b4775-5a38-4d3a-a644-8c5357c508ce None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:12.797865 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 185/367] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:15:12 2022] GET /identity/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:12.799342 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b4775-5a38-4d3a-a644-8c5357c508ce None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:12.799638 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 183/368] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:15:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:12.802760 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86c1665-27b7-45d2-b913-b084f7aaf16a None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:12.897324 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:12.916343 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:12.927196 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86c1665-27b7-45d2-b913-b084f7aaf16a None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:12.935806 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b4775-5a38-4d3a-a644-8c5357c508ce None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:12.981182 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32592b-9603-499b-adfb-61944e4f034f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:12.982140 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32592b-9603-499b-adfb-61944e4f034f None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:12.982424 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32592b-9603-499b-adfb-61944e4f034f None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:12.982697 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32592b-9603-499b-adfb-61944e4f034f None admin] PATH_INFO: `/v3/groups/dd675b91e3ef45dc8650e456e0529fd7` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:12.984106 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e32592b-9603-499b-adfb-61944e4f034f None admin] RBAC: Authorizing `identity:get_group(group_id=dd675b91e3ef45dc8650e456e0529fd7)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:12.985673 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e32592b-9603-499b-adfb-61944e4f034f None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:12.986289 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4abe0-565b-4420-9884-9cf9d52fc045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:12.987317 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4abe0-565b-4420-9884-9cf9d52fc045 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:12.987767 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 184/369] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Jul 7 15:15:12 2022] GET /identity/v3/groups/dd675b91e3ef45dc8650e456e0529fd7 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:15:12.988049 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4abe0-565b-4420-9884-9cf9d52fc045 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:12.988605 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32592b-9603-499b-adfb-61944e4f034f None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:12.988843 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4abe0-565b-4420-9884-9cf9d52fc045 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:13.003759 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f4abe0-565b-4420-9884-9cf9d52fc045 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:13.014593 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-42f4abe0-565b-4420-9884-9cf9d52fc045 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:15:13.014593 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:13.014593 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:13.014593 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:13.014593 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:13.014593 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:13.014593 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:13.014593 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:13.014593 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:13.014593 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:13.014593 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:13.014593 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:13.014593 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:13.014593 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:13.014593 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:13.014593 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:13.015503 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:13.015503 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:13.015503 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:13.015503 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:13.015503 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:13.015503 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:13.015503 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:13.015503 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:13.015503 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:13.015503 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:13.015503 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:13.015503 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:13.015503 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:13.015503 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:13.015503 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:13.015503 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:13.016917 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:13.016917 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:13.016917 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:13.016917 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:13.016917 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:13.016917 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:13.016917 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:13.016917 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:13.016917 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:13.016917 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:13.016917 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:15:13.016917 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:13.024118 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 186/370] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Jul 7 15:15:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:13.024118 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4abe0-565b-4420-9884-9cf9d52fc045 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:13.156087 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32592b-9603-499b-adfb-61944e4f034f None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:13.183096 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4abe0-565b-4420-9884-9cf9d52fc045 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:13.196149 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a669a4-dfc3-4aa2-956b-52fddc3d1a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:13.196881 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a669a4-dfc3-4aa2-956b-52fddc3d1a1e None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:13.197231 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a669a4-dfc3-4aa2-956b-52fddc3d1a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:13.197573 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a669a4-dfc3-4aa2-956b-52fddc3d1a1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:13.211954 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:13.224381 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3268ecf4-baa5-41e4-8b53-2db844adb281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:13.228218 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3268ecf4-baa5-41e4-8b53-2db844adb281 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:13.228218 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3268ecf4-baa5-41e4-8b53-2db844adb281 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:13.228218 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3268ecf4-baa5-41e4-8b53-2db844adb281 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:13.228218 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3268ecf4-baa5-41e4-8b53-2db844adb281 None admin] RBAC: Adding query filter params (group.id=dd675b91e3ef45dc8650e456e0529fd7, role.id=d43d0d189e254500875f5f86e6abd178, scope.project.id=a6fde692f8db4d7eb4858942b0b6e5bf) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:13.228218 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3268ecf4-baa5-41e4-8b53-2db844adb281 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:13.229546 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3268ecf4-baa5-41e4-8b53-2db844adb281 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:13.238006 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 187/371] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Jul 7 15:15:13 2022] GET /identity/v3/role_assignments?group.id=dd675b91e3ef45dc8650e456e0529fd7&scope.project.id=a6fde692f8db4d7eb4858942b0b6e5bf&role.id=d43d0d189e254500875f5f86e6abd178 => generated 260 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:13.238833 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3268ecf4-baa5-41e4-8b53-2db844adb281 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:13.263996 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:13.332851 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a669a4-dfc3-4aa2-956b-52fddc3d1a1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:13.334355 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a669a4-dfc3-4aa2-956b-52fddc3d1a1e None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:13.385412 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3268ecf4-baa5-41e4-8b53-2db844adb281 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:13.442149 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691f8cc-0bcc-4754-a298-0c32de599f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:13.443109 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691f8cc-0bcc-4754-a298-0c32de599f48 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:13.443596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691f8cc-0bcc-4754-a298-0c32de599f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:13.443998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691f8cc-0bcc-4754-a298-0c32de599f48 None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:13.446400 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4691f8cc-0bcc-4754-a298-0c32de599f48 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:13.446868 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4691f8cc-0bcc-4754-a298-0c32de599f48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:13.449107 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4691f8cc-0bcc-4754-a298-0c32de599f48 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:13.467408 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 188/372] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:15:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:13.475340 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691f8cc-0bcc-4754-a298-0c32de599f48 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:13.542033 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 185/373] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:15:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 15:15:13.548386 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a669a4-dfc3-4aa2-956b-52fddc3d1a1e None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:13.588900 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:13.635493 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691f8cc-0bcc-4754-a298-0c32de599f48 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:13.639957 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:13.686734 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e4f07-18e6-4ed6-beb2-16c1d9b49861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:13.687668 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e4f07-18e6-4ed6-beb2-16c1d9b49861 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:13.688069 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e4f07-18e6-4ed6-beb2-16c1d9b49861 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:13.689054 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e4f07-18e6-4ed6-beb2-16c1d9b49861 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:13.704273 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a669a4-dfc3-4aa2-956b-52fddc3d1a1e None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:13.742947 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b1d239-3a4e-46a7-b5a7-c4dd0e07781c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:13.743847 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b1d239-3a4e-46a7-b5a7-c4dd0e07781c None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:13.744217 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b1d239-3a4e-46a7-b5a7-c4dd0e07781c None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:13.744501 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b1d239-3a4e-46a7-b5a7-c4dd0e07781c None admin] PATH_INFO: `/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:13.745800 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b1d239-3a4e-46a7-b5a7-c4dd0e07781c None admin] RBAC: Authorizing `identity:get_project(project_id=a6fde692f8db4d7eb4858942b0b6e5bf)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:13.747326 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b1d239-3a4e-46a7-b5a7-c4dd0e07781c None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:13.749507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 186/374] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:15:13 2022] GET /identity/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:13.755585 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b1d239-3a4e-46a7-b5a7-c4dd0e07781c None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:13.799100 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e4f07-18e6-4ed6-beb2-16c1d9b49861 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:13.800848 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e4f07-18e6-4ed6-beb2-16c1d9b49861 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:13.911035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:13.942881 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b1d239-3a4e-46a7-b5a7-c4dd0e07781c None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:13.954452 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:13.982668 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d0147c-7cf9-43e2-8d66-4f4728332d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:13.983532 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d0147c-7cf9-43e2-8d66-4f4728332d81 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:13.983865 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d0147c-7cf9-43e2-8d66-4f4728332d81 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:13.984143 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d0147c-7cf9-43e2-8d66-4f4728332d81 None admin] PATH_INFO: `/v3/groups/dd675b91e3ef45dc8650e456e0529fd7` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:13.984246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 189/375] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:15:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 15:15:13.986255 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d0147c-7cf9-43e2-8d66-4f4728332d81 None admin] RBAC: Authorizing `identity:get_group(group_id=dd675b91e3ef45dc8650e456e0529fd7)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:13.988187 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d0147c-7cf9-43e2-8d66-4f4728332d81 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:13.990345 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 187/376] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Jul 7 15:15:13 2022] GET /identity/v3/groups/dd675b91e3ef45dc8650e456e0529fd7 => generated 226 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:13.991084 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e4f07-18e6-4ed6-beb2-16c1d9b49861 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:14.006057 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d0147c-7cf9-43e2-8d66-4f4728332d81 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:14.132367 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d0147c-7cf9-43e2-8d66-4f4728332d81 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:14.169881 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc572c3-8969-4466-ab17-bcb0ec769338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:14.170539 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc572c3-8969-4466-ab17-bcb0ec769338 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:14.170824 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc572c3-8969-4466-ab17-bcb0ec769338 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:14.171101 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc572c3-8969-4466-ab17-bcb0ec769338 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:14.171972 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc572c3-8969-4466-ab17-bcb0ec769338 None admin] RBAC: Adding query filter params (group.id=dd675b91e3ef45dc8650e456e0529fd7, role.id=c82203e4734346719cafb16230f45942, scope.project.id=a6fde692f8db4d7eb4858942b0b6e5bf) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:14.172284 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc572c3-8969-4466-ab17-bcb0ec769338 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:14.174400 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc572c3-8969-4466-ab17-bcb0ec769338 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:14.180875 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 188/377] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Jul 7 15:15:14 2022] GET /identity/v3/role_assignments?group.id=dd675b91e3ef45dc8650e456e0529fd7&scope.project.id=a6fde692f8db4d7eb4858942b0b6e5bf&role.id=c82203e4734346719cafb16230f45942 => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:15:14.226712 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e4f07-18e6-4ed6-beb2-16c1d9b49861 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:14.278701 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a18319-ed15-4a33-802a-66a985956298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:14.279623 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a18319-ed15-4a33-802a-66a985956298 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:14.279863 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a18319-ed15-4a33-802a-66a985956298 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:14.280138 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a18319-ed15-4a33-802a-66a985956298 None admin] PATH_INFO: `/v3/projects/28133f9a98be4af88ed357f76f5a4a2e` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:14.281317 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a18319-ed15-4a33-802a-66a985956298 None admin] RBAC: Authorizing `identity:get_project(project_id=28133f9a98be4af88ed357f76f5a4a2e)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:14.283317 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a18319-ed15-4a33-802a-66a985956298 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:14.286355 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 190/378] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:15:13 2022] GET /identity/v3/projects/28133f9a98be4af88ed357f76f5a4a2e => generated 344 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:14.290916 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc572c3-8969-4466-ab17-bcb0ec769338 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:14.361419 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:14.425043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982da68-be71-4787-aedf-565c16840dd8 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:14.425380 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982da68-be71-4787-aedf-565c16840dd8 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:14.425595 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982da68-be71-4787-aedf-565c16840dd8 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:14.430978 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc572c3-8969-4466-ab17-bcb0ec769338 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:14.431144 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 191/379] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:14.432430 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c6a4c0-4841-41ba-ab69-6cf9e922664d None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:14.432813 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c6a4c0-4841-41ba-ab69-6cf9e922664d None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:14.433094 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c6a4c0-4841-41ba-ab69-6cf9e922664d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:14.463238 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-e1c6a4c0-4841-41ba-ab69-6cf9e922664d None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:14.464561 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-e1c6a4c0-4841-41ba-ab69-6cf9e922664d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:14.479523 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448efde-e2e0-4c89-b17f-1fe48cb34ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:14.480418 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448efde-e2e0-4c89-b17f-1fe48cb34ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:14.480698 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448efde-e2e0-4c89-b17f-1fe48cb34ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:14.480971 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448efde-e2e0-4c89-b17f-1fe48cb34ff5 None admin] PATH_INFO: `/v3/groups/8ccaec72b7db46078702143bc510dafa` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:14.492576 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e448efde-e2e0-4c89-b17f-1fe48cb34ff5 None admin] RBAC: Authorizing `identity:get_group(group_id=8ccaec72b7db46078702143bc510dafa)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:14.494088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e448efde-e2e0-4c89-b17f-1fe48cb34ff5 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:14.496135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 189/380] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Jul 7 15:15:14 2022] GET /identity/v3/groups/8ccaec72b7db46078702143bc510dafa => generated 229 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:14.505115 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448efde-e2e0-4c89-b17f-1fe48cb34ff5 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:14.533937 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:14.602682 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 192/381] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:14.628420 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c6a4c0-4841-41ba-ab69-6cf9e922664d None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:14.633640 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448efde-e2e0-4c89-b17f-1fe48cb34ff5 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:14.642658 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:14.679007 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e6606-1d8a-4fd1-9c66-4c93d854bdcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:14.680410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e6606-1d8a-4fd1-9c66-4c93d854bdcb None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:14.680800 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e6606-1d8a-4fd1-9c66-4c93d854bdcb None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:14.681177 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e6606-1d8a-4fd1-9c66-4c93d854bdcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:14.682180 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6e6606-1d8a-4fd1-9c66-4c93d854bdcb None admin] RBAC: Adding query filter params (group.id=8ccaec72b7db46078702143bc510dafa, role.id=0e7793fbf3064037984da651879e3a94, scope.project.id=28133f9a98be4af88ed357f76f5a4a2e) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:14.682596 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6e6606-1d8a-4fd1-9c66-4c93d854bdcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:14.684693 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6e6606-1d8a-4fd1-9c66-4c93d854bdcb None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:14.693897 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 190/382] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Jul 7 15:15:14 2022] GET /identity/v3/role_assignments?group.id=8ccaec72b7db46078702143bc510dafa&scope.project.id=28133f9a98be4af88ed357f76f5a4a2e&role.id=0e7793fbf3064037984da651879e3a94 => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:15:14.761054 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c6a4c0-4841-41ba-ab69-6cf9e922664d None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:14.801573 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6e35c-7756-4ac2-a0b4-c5d52244e220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:14.802384 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6e35c-7756-4ac2-a0b4-c5d52244e220 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:14.802662 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6e35c-7756-4ac2-a0b4-c5d52244e220 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:14.802934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6e35c-7756-4ac2-a0b4-c5d52244e220 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:14.815142 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae6e35c-7756-4ac2-a0b4-c5d52244e220 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:14.823259 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-5ae6e35c-7756-4ac2-a0b4-c5d52244e220 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 07 15:15:14.823259 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:14.823259 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:14.823259 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:14.823259 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:14.823259 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:14.823259 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:14.823259 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:14.823259 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:14.823259 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:14.823259 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:14.823259 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:14.823259 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:14.823259 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:14.823259 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:14.823259 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:14.824260 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:14.824260 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:14.824260 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:14.824260 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:14.824260 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:14.824260 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:14.824260 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:14.824260 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:14.824260 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:14.824260 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:14.824260 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:14.824260 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:14.824260 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:14.824260 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:14.824260 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:14.824260 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:14.826203 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:14.826203 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:14.826203 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:14.826203 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:14.826203 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:14.826203 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:14.826203 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:14.826203 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:14.826203 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:14.826203 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:14.826203 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 07 15:15:14.826203 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:14.826203 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 193/383] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:15:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:15:14.831188 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e6606-1d8a-4fd1-9c66-4c93d854bdcb None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:14.944721 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e6606-1d8a-4fd1-9c66-4c93d854bdcb None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:15.022555 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a57f2-497f-4abf-9584-647db114a2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:15.023479 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a57f2-497f-4abf-9584-647db114a2fa None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:15.024005 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a57f2-497f-4abf-9584-647db114a2fa None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:15.024433 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a57f2-497f-4abf-9584-647db114a2fa None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:15.025426 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194a57f2-497f-4abf-9584-647db114a2fa None admin] RBAC: Adding query filter params (name=member) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:15.025887 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194a57f2-497f-4abf-9584-647db114a2fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:15.027428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194a57f2-497f-4abf-9584-647db114a2fa None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:15.037347 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 191/384] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Jul 7 15:15:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:15.042956 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6e35c-7756-4ac2-a0b4-c5d52244e220 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:15.079684 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:15.153420 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6e35c-7756-4ac2-a0b4-c5d52244e220 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:15.195964 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5035e0a5-c566-4a5c-abe3-bcd8e8beff62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:15.197144 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035e0a5-c566-4a5c-abe3-bcd8e8beff62 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:15.197580 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035e0a5-c566-4a5c-abe3-bcd8e8beff62 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:15.198043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035e0a5-c566-4a5c-abe3-bcd8e8beff62 None admin] PATH_INFO: `/v3/groups/dd675b91e3ef45dc8650e456e0529fd7` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:15.199353 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5035e0a5-c566-4a5c-abe3-bcd8e8beff62 None admin] RBAC: Authorizing `identity:get_group(group_id=dd675b91e3ef45dc8650e456e0529fd7)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:15.200973 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5035e0a5-c566-4a5c-abe3-bcd8e8beff62 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:15.203040 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 194/385] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Jul 7 15:15:15 2022] GET /identity/v3/groups/dd675b91e3ef45dc8650e456e0529fd7 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:15.208704 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a57f2-497f-4abf-9584-647db114a2fa None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:15.216299 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ede70d-2d58-452d-9349-d01e4afeb3f2 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:15.216771 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ede70d-2d58-452d-9349-d01e4afeb3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:15.217216 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ede70d-2d58-452d-9349-d01e4afeb3f2 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:15.218963 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 195/386] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:15:15.224075 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c8d38-f61b-48dc-907a-18ff237ecee6 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:15.224365 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c8d38-f61b-48dc-907a-18ff237ecee6 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:15.224641 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c8d38-f61b-48dc-907a-18ff237ecee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:15.242192 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-733c8d38-f61b-48dc-907a-18ff237ecee6 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:15.243664 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-733c8d38-f61b-48dc-907a-18ff237ecee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:15.270903 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:15.355936 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 196/387] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 15:15:15.385402 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a57f2-497f-4abf-9584-647db114a2fa None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:15.420020 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c8d38-f61b-48dc-907a-18ff237ecee6 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:15.433594 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614eefa6-36a8-4764-9f38-abd5265bf406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:15.434756 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614eefa6-36a8-4764-9f38-abd5265bf406 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:15.435221 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614eefa6-36a8-4764-9f38-abd5265bf406 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:15.435698 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614eefa6-36a8-4764-9f38-abd5265bf406 None admin] PATH_INFO: `/v3/projects/db427256e2ec4ec084f762a615d0f187` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:15.440303 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614eefa6-36a8-4764-9f38-abd5265bf406 None admin] RBAC: Authorizing `identity:get_project(project_id=db427256e2ec4ec084f762a615d0f187)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:15.442188 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614eefa6-36a8-4764-9f38-abd5265bf406 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:15.444436 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 192/388] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:15:15 2022] GET /identity/v3/projects/db427256e2ec4ec084f762a615d0f187 => generated 298 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:15.452614 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614eefa6-36a8-4764-9f38-abd5265bf406 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:15.456883 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:15.585759 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614eefa6-36a8-4764-9f38-abd5265bf406 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:15.590067 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:15.623075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f26a8f-69ea-497e-ac96-c3e25ff969d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:15.623958 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f26a8f-69ea-497e-ac96-c3e25ff969d5 None admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:15.624282 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f26a8f-69ea-497e-ac96-c3e25ff969d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:15.624564 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f26a8f-69ea-497e-ac96-c3e25ff969d5 None admin] PATH_INFO: `/v3/projects/db427256e2ec4ec084f762a615d0f187/groups/dd675b91e3ef45dc8650e456e0529fd7/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:15.627188 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f26a8f-69ea-497e-ac96-c3e25ff969d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=db427256e2ec4ec084f762a615d0f187, group_id=dd675b91e3ef45dc8650e456e0529fd7, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:15.628868 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f26a8f-69ea-497e-ac96-c3e25ff969d5 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:15.647896 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c8d38-f61b-48dc-907a-18ff237ecee6 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:15.683169 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6e6c7-0cd2-4cae-b88f-f5e07dc7d070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:15.684071 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6e6c7-0cd2-4cae-b88f-f5e07dc7d070 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:15.684348 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6e6c7-0cd2-4cae-b88f-f5e07dc7d070 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:15.684615 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6e6c7-0cd2-4cae-b88f-f5e07dc7d070 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:15.701643 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 193/389] 10.222.0.25 () {58 vars in 1442 bytes} [Thu Jul 7 15:15:15 2022] PUT /identity/v3/projects/db427256e2ec4ec084f762a615d0f187/groups/dd675b91e3ef45dc8650e456e0529fd7/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:15:15.703156 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c6e6c7-0cd2-4cae-b88f-f5e07dc7d070 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:15.712516 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-41c6e6c7-0cd2-4cae-b88f-f5e07dc7d070 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 15:15:15.712516 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:15.712516 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:15.712516 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:15.712516 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:15.712516 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:15.712516 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:15.712516 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:15.712516 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:15.712516 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:15.712516 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:15.712516 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:15.712516 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:15.712516 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:15.712516 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:15.712516 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:15.714231 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:15.714231 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:15.714231 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:15.714231 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:15.714231 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:15.714231 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:15.714231 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:15.714231 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:15.714231 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:15.714231 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:15.714231 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:15.714231 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:15.714231 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:15.714231 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:15.714231 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:15.714231 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:15.715043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:15.715043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:15.715043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:15.715043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:15.715043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:15.715043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:15.715043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:15.715043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:15.715043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:15.715043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:15.715043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 15:15:15.715043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:15.715043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 197/390] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Jul 7 15:15:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:15:15.722263 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f26a8f-69ea-497e-ac96-c3e25ff969d5 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:15.832216 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f26a8f-69ea-497e-ac96-c3e25ff969d5 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:15.885744 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73865a-78ac-4b5a-bf2a-6473ecaac598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:15.886342 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73865a-78ac-4b5a-bf2a-6473ecaac598 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:15.886704 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73865a-78ac-4b5a-bf2a-6473ecaac598 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:15.886989 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73865a-78ac-4b5a-bf2a-6473ecaac598 None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:15.887774 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f73865a-78ac-4b5a-bf2a-6473ecaac598 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:15.888082 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f73865a-78ac-4b5a-bf2a-6473ecaac598 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:15.889486 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f73865a-78ac-4b5a-bf2a-6473ecaac598 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:15.903817 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 194/391] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:15:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:15.909488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6e6c7-0cd2-4cae-b88f-f5e07dc7d070 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:16.030384 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:16.042986 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6e6c7-0cd2-4cae-b88f-f5e07dc7d070 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:16.078762 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293be137-469a-4797-92b9-3e31157b116f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:16.079647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293be137-469a-4797-92b9-3e31157b116f None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:16.079932 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293be137-469a-4797-92b9-3e31157b116f None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:16.080208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293be137-469a-4797-92b9-3e31157b116f None admin] PATH_INFO: `/v3/groups/dd675b91e3ef45dc8650e456e0529fd7` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:16.081374 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293be137-469a-4797-92b9-3e31157b116f None admin] RBAC: Authorizing `identity:get_group(group_id=dd675b91e3ef45dc8650e456e0529fd7)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:16.082812 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293be137-469a-4797-92b9-3e31157b116f None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:16.089993 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73865a-78ac-4b5a-bf2a-6473ecaac598 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:16.091058 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 198/392] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Jul 7 15:15:15 2022] GET /identity/v3/groups/dd675b91e3ef45dc8650e456e0529fd7 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:16.104880 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd62b7-f532-4889-a431-d7945358ed6d None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:16.105183 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd62b7-f532-4889-a431-d7945358ed6d None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:16.105438 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd62b7-f532-4889-a431-d7945358ed6d None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:16.106787 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 199/393] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:15:16.112217 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe366ce-6460-4c87-86ac-1dfb96deeb0e None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:16.112507 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe366ce-6460-4c87-86ac-1dfb96deeb0e None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:16.112788 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe366ce-6460-4c87-86ac-1dfb96deeb0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:16.129348 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-dfe366ce-6460-4c87-86ac-1dfb96deeb0e None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:16.130543 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-dfe366ce-6460-4c87-86ac-1dfb96deeb0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:16.139127 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:16.253559 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 200/394] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 15:15:16.266511 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73865a-78ac-4b5a-bf2a-6473ecaac598 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:16.285912 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe366ce-6460-4c87-86ac-1dfb96deeb0e None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:16.326394 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03df0976-69ab-4c92-bf06-0ba7f0308647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:16.327262 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df0976-69ab-4c92-bf06-0ba7f0308647 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:16.327537 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df0976-69ab-4c92-bf06-0ba7f0308647 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:16.327834 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df0976-69ab-4c92-bf06-0ba7f0308647 None admin] PATH_INFO: `/v3/projects/db427256e2ec4ec084f762a615d0f187` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:16.329672 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03df0976-69ab-4c92-bf06-0ba7f0308647 None admin] RBAC: Authorizing `identity:get_project(project_id=db427256e2ec4ec084f762a615d0f187)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:16.331091 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03df0976-69ab-4c92-bf06-0ba7f0308647 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:16.332979 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 195/395] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:15:16 2022] GET /identity/v3/projects/db427256e2ec4ec084f762a615d0f187 => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:16.339030 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03df0976-69ab-4c92-bf06-0ba7f0308647 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:16.427032 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:16.452767 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe366ce-6460-4c87-86ac-1dfb96deeb0e None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:16.513068 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b7e4d9-72cd-46fd-a556-1d9571e4cd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:16.513868 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b7e4d9-72cd-46fd-a556-1d9571e4cd03 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:16.514151 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b7e4d9-72cd-46fd-a556-1d9571e4cd03 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:16.514428 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b7e4d9-72cd-46fd-a556-1d9571e4cd03 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:16.517901 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:16.533109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03df0976-69ab-4c92-bf06-0ba7f0308647 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:16.535020 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b7e4d9-72cd-46fd-a556-1d9571e4cd03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:16.542841 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-97b7e4d9-72cd-46fd-a556-1d9571e4cd03 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 07 15:15:16.542841 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:16.542841 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:16.542841 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:16.542841 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:16.542841 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:16.542841 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:16.542841 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:16.542841 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:16.542841 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:16.542841 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:16.542841 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:16.542841 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:16.542841 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:16.542841 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:16.542841 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:16.543783 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:16.543783 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:16.543783 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:16.543783 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:16.543783 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:16.543783 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:16.543783 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:16.543783 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:16.543783 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:16.543783 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:16.543783 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:16.543783 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:16.543783 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:16.543783 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:16.543783 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:16.543783 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:16.544581 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:16.544581 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:16.544581 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:16.544581 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:16.544581 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:16.544581 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:16.544581 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:16.544581 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:16.544581 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:16.544581 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:16.544581 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 07 15:15:16.544581 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:16.555297 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 201/396] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:15:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:15:16.557861 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b7e4d9-72cd-46fd-a556-1d9571e4cd03 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:16.608523 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921dd7e-c28c-4fbc-a92c-d28eed45c0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:16.611155 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921dd7e-c28c-4fbc-a92c-d28eed45c0b9 None admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:16.612552 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921dd7e-c28c-4fbc-a92c-d28eed45c0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:16.613045 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921dd7e-c28c-4fbc-a92c-d28eed45c0b9 None admin] PATH_INFO: `/v3/projects/db427256e2ec4ec084f762a615d0f187/groups/dd675b91e3ef45dc8650e456e0529fd7/roles/c82203e4734346719cafb16230f45942` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:16.617836 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921dd7e-c28c-4fbc-a92c-d28eed45c0b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=db427256e2ec4ec084f762a615d0f187, group_id=dd675b91e3ef45dc8650e456e0529fd7, role_id=c82203e4734346719cafb16230f45942)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:16.620374 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921dd7e-c28c-4fbc-a92c-d28eed45c0b9 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:16.698117 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b7e4d9-72cd-46fd-a556-1d9571e4cd03 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:16.725312 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 196/397] 10.222.0.25 () {58 vars in 1442 bytes} [Thu Jul 7 15:15:16 2022] PUT /identity/v3/projects/db427256e2ec4ec084f762a615d0f187/groups/dd675b91e3ef45dc8650e456e0529fd7/roles/c82203e4734346719cafb16230f45942 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:15:16.749588 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f3164-36e9-4bf0-acca-0a3bbd778470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:16.750677 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f3164-36e9-4bf0-acca-0a3bbd778470 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:16.751268 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f3164-36e9-4bf0-acca-0a3bbd778470 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:16.751792 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f3164-36e9-4bf0-acca-0a3bbd778470 None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:16.752762 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f3164-36e9-4bf0-acca-0a3bbd778470 None admin] RBAC: Adding query filter params (name=member) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:16.753283 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f3164-36e9-4bf0-acca-0a3bbd778470 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:16.754941 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f3164-36e9-4bf0-acca-0a3bbd778470 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:16.766637 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 202/398] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Jul 7 15:15:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:16.773299 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921dd7e-c28c-4fbc-a92c-d28eed45c0b9 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:16.904441 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0548e093-a54f-4747-81dd-76288526bda8 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:16.905120 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0548e093-a54f-4747-81dd-76288526bda8 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:16.905631 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0548e093-a54f-4747-81dd-76288526bda8 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:16.907286 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 203/399] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:15:16.911516 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921dd7e-c28c-4fbc-a92c-d28eed45c0b9 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:16.913870 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66652fd-5fd0-4100-80f1-a79c21058989 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:16.914464 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66652fd-5fd0-4100-80f1-a79c21058989 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:16.922056 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66652fd-5fd0-4100-80f1-a79c21058989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:16.943321 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-c66652fd-5fd0-4100-80f1-a79c21058989 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:16.945124 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-c66652fd-5fd0-4100-80f1-a79c21058989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:16.949345 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:17.002473 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92caff3-0bb5-41cf-9323-b3b7db614f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:17.003340 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92caff3-0bb5-41cf-9323-b3b7db614f5c None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:17.003821 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92caff3-0bb5-41cf-9323-b3b7db614f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:17.004102 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92caff3-0bb5-41cf-9323-b3b7db614f5c None admin] PATH_INFO: `/v3/groups/dd675b91e3ef45dc8650e456e0529fd7` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:17.005860 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92caff3-0bb5-41cf-9323-b3b7db614f5c None admin] RBAC: Authorizing `identity:get_group(group_id=dd675b91e3ef45dc8650e456e0529fd7)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:17.007592 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92caff3-0bb5-41cf-9323-b3b7db614f5c None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:17.009848 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 197/400] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Jul 7 15:15:16 2022] GET /identity/v3/groups/dd675b91e3ef45dc8650e456e0529fd7 => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:15:17.015559 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92caff3-0bb5-41cf-9323-b3b7db614f5c None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:17.032351 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 204/401] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 15:15:17.039597 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:17.057789 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66652fd-5fd0-4100-80f1-a79c21058989 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:17.184043 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92caff3-0bb5-41cf-9323-b3b7db614f5c None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:17.237824 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66652fd-5fd0-4100-80f1-a79c21058989 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:17.256879 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba1807-a1a9-4a2b-8d98-d0c86ab746dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:17.257757 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba1807-a1a9-4a2b-8d98-d0c86ab746dd None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:17.258037 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba1807-a1a9-4a2b-8d98-d0c86ab746dd None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:17.258309 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba1807-a1a9-4a2b-8d98-d0c86ab746dd None admin] PATH_INFO: `/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:17.259516 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfba1807-a1a9-4a2b-8d98-d0c86ab746dd None admin] RBAC: Authorizing `identity:get_project(project_id=a6fde692f8db4d7eb4858942b0b6e5bf)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:17.260979 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfba1807-a1a9-4a2b-8d98-d0c86ab746dd None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:17.262744 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 198/402] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:15:17 2022] GET /identity/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:17.280386 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba1807-a1a9-4a2b-8d98-d0c86ab746dd None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:17.295836 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04497699-01be-419d-9e46-8a3fc694fed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:17.297067 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04497699-01be-419d-9e46-8a3fc694fed0 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:17.298142 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04497699-01be-419d-9e46-8a3fc694fed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:17.298619 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04497699-01be-419d-9e46-8a3fc694fed0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:17.314226 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:17.324495 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04497699-01be-419d-9e46-8a3fc694fed0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:17.336031 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-04497699-01be-419d-9e46-8a3fc694fed0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 15:15:17.336031 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:17.336031 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:17.336031 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:17.336031 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:17.336031 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:17.336031 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:17.336031 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:17.336031 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:17.336031 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:17.336031 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:17.336031 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:17.336031 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:17.336031 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:17.336031 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:17.336031 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:17.336947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:17.336947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:17.336947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:17.336947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:17.336947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:17.336947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:17.336947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:17.336947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:17.336947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:17.336947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:17.336947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:17.336947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:17.336947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:17.336947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:17.336947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:17.336947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:17.337760 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:17.337760 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:17.337760 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:17.337760 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:17.337760 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:17.337760 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:17.337760 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:17.337760 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:17.337760 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:17.337760 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:17.337760 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 15:15:17.337760 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:17.347498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 205/403] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Jul 7 15:15:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:15:17.351148 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04497699-01be-419d-9e46-8a3fc694fed0 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:17.429382 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:17.444931 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba1807-a1a9-4a2b-8d98-d0c86ab746dd None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:17.469052 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04497699-01be-419d-9e46-8a3fc694fed0 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:17.479473 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f08f5c-67ab-4e1a-8b54-50602e4b257d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:17.480375 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f08f5c-67ab-4e1a-8b54-50602e4b257d None admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:17.480655 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f08f5c-67ab-4e1a-8b54-50602e4b257d None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:17.480928 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f08f5c-67ab-4e1a-8b54-50602e4b257d None admin] PATH_INFO: `/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf/groups/dd675b91e3ef45dc8650e456e0529fd7/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:17.482886 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f08f5c-67ab-4e1a-8b54-50602e4b257d None admin] RBAC: Authorizing `identity:create_grant(project_id=a6fde692f8db4d7eb4858942b0b6e5bf, group_id=dd675b91e3ef45dc8650e456e0529fd7, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:17.484328 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f08f5c-67ab-4e1a-8b54-50602e4b257d None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:17.506680 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9505174f-c1a5-492e-aac6-c20688e4e7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:17.507269 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9505174f-c1a5-492e-aac6-c20688e4e7c4 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:17.507547 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9505174f-c1a5-492e-aac6-c20688e4e7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:17.507841 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9505174f-c1a5-492e-aac6-c20688e4e7c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:17.508582 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9505174f-c1a5-492e-aac6-c20688e4e7c4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:17.508885 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9505174f-c1a5-492e-aac6-c20688e4e7c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:17.510235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9505174f-c1a5-492e-aac6-c20688e4e7c4 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:17.518570 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 206/404] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:15:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:17.522695 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80059af6-3f31-4ac5-8021-fc9ca755992a None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:17.522695 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80059af6-3f31-4ac5-8021-fc9ca755992a None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:17.522695 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80059af6-3f31-4ac5-8021-fc9ca755992a None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:17.522695 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 207/405] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:17.526047 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80059af6-3f31-4ac5-8021-fc9ca755992a None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:17.599592 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:17.599992 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 199/406] 10.222.0.25 () {58 vars in 1442 bytes} [Thu Jul 7 15:15:17 2022] PUT /identity/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf/groups/dd675b91e3ef45dc8650e456e0529fd7/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:15:17.601772 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d300a741-dee8-488c-8f2f-e43e734993d4 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:17.602270 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d300a741-dee8-488c-8f2f-e43e734993d4 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:17.602777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d300a741-dee8-488c-8f2f-e43e734993d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:17.644979 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-d300a741-dee8-488c-8f2f-e43e734993d4 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:17.646771 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-d300a741-dee8-488c-8f2f-e43e734993d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:17.648418 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80059af6-3f31-4ac5-8021-fc9ca755992a None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:17.694127 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989db7c9-d961-4edf-8e5a-70970e18659c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:17.695335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989db7c9-d961-4edf-8e5a-70970e18659c None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:17.695636 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989db7c9-d961-4edf-8e5a-70970e18659c None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:17.695913 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989db7c9-d961-4edf-8e5a-70970e18659c None admin] PATH_INFO: `/v3/groups/dd675b91e3ef45dc8650e456e0529fd7` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:17.697251 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989db7c9-d961-4edf-8e5a-70970e18659c None admin] RBAC: Authorizing `identity:get_group(group_id=dd675b91e3ef45dc8650e456e0529fd7)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:17.699131 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989db7c9-d961-4edf-8e5a-70970e18659c None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:17.701187 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 208/407] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Jul 7 15:15:17 2022] GET /identity/v3/groups/dd675b91e3ef45dc8650e456e0529fd7 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:17.708791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989db7c9-d961-4edf-8e5a-70970e18659c None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:17.805916 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 200/408] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:17.845463 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989db7c9-d961-4edf-8e5a-70970e18659c None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:17.854847 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d300a741-dee8-488c-8f2f-e43e734993d4 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:17.876382 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:17.887639 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e9470-8beb-4447-b900-3460269e3248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:17.888820 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e9470-8beb-4447-b900-3460269e3248 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:17.889318 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e9470-8beb-4447-b900-3460269e3248 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:17.889797 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e9470-8beb-4447-b900-3460269e3248 None admin] PATH_INFO: `/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:17.891254 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375e9470-8beb-4447-b900-3460269e3248 None admin] RBAC: Authorizing `identity:get_project(project_id=a6fde692f8db4d7eb4858942b0b6e5bf)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:17.893050 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375e9470-8beb-4447-b900-3460269e3248 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:17.895360 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 209/409] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:15:17 2022] GET /identity/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:17.901244 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e9470-8beb-4447-b900-3460269e3248 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:17.957820 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:17.984889 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d300a741-dee8-488c-8f2f-e43e734993d4 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:18.057616 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1febf62-ff8a-421d-b72f-e6364f01de46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:18.058762 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1febf62-ff8a-421d-b72f-e6364f01de46 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:18.059243 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1febf62-ff8a-421d-b72f-e6364f01de46 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:18.060914 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1febf62-ff8a-421d-b72f-e6364f01de46 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:18.075486 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e9470-8beb-4447-b900-3460269e3248 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:18.088339 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1febf62-ff8a-421d-b72f-e6364f01de46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:18.100683 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-e1febf62-ff8a-421d-b72f-e6364f01de46 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:15:18.100683 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:18.100683 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:18.100683 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:18.100683 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:18.100683 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:18.100683 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:18.100683 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:18.100683 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:18.100683 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:18.100683 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:18.100683 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:18.100683 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:18.100683 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:18.100683 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:18.100683 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:18.101663 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:18.101663 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:18.101663 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:18.101663 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:18.101663 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:18.101663 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:18.101663 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:18.101663 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:18.101663 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:18.101663 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:18.101663 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:18.101663 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:18.101663 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:18.101663 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:18.101663 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:18.101663 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:18.102482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:18.102482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:18.102482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:18.102482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:18.102482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:18.102482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:18.102482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:18.102482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:18.102482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:18.102482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:18.102482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:15:18.102482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:18.102482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 201/410] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Jul 7 15:15:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:18.121908 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1febf62-ff8a-421d-b72f-e6364f01de46 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:18.131138 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd01a1b-24a7-47b5-b2cb-7d4d5716b206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:18.132512 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd01a1b-24a7-47b5-b2cb-7d4d5716b206 None admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:18.135924 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd01a1b-24a7-47b5-b2cb-7d4d5716b206 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:18.136368 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd01a1b-24a7-47b5-b2cb-7d4d5716b206 None admin] PATH_INFO: `/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf/groups/dd675b91e3ef45dc8650e456e0529fd7/roles/c82203e4734346719cafb16230f45942` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:18.140979 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd01a1b-24a7-47b5-b2cb-7d4d5716b206 None admin] RBAC: Authorizing `identity:create_grant(project_id=a6fde692f8db4d7eb4858942b0b6e5bf, group_id=dd675b91e3ef45dc8650e456e0529fd7, role_id=c82203e4734346719cafb16230f45942)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:18.143174 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd01a1b-24a7-47b5-b2cb-7d4d5716b206 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:18.225769 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 210/411] 10.222.0.25 () {58 vars in 1442 bytes} [Thu Jul 7 15:15:17 2022] PUT /identity/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf/groups/dd675b91e3ef45dc8650e456e0529fd7/roles/c82203e4734346719cafb16230f45942 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:15:18.225769 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4b45f-1816-4539-a3f9-a72528b63683 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:18.225769 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4b45f-1816-4539-a3f9-a72528b63683 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:18.225769 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4b45f-1816-4539-a3f9-a72528b63683 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:18.226243 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 211/412] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:18.233088 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb15760-0de6-4939-a676-37a322b0d4fc None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:18.233475 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb15760-0de6-4939-a676-37a322b0d4fc None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:18.233851 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb15760-0de6-4939-a676-37a322b0d4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:18.255178 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-abb15760-0de6-4939-a676-37a322b0d4fc None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:18.257227 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-abb15760-0de6-4939-a676-37a322b0d4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:18.277618 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1febf62-ff8a-421d-b72f-e6364f01de46 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:18.324271 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:18.345695 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 212/413] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 15:15:18.364396 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6921e02e-d1ce-4b7a-b0c6-ee49868ad001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:18.365339 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921e02e-d1ce-4b7a-b0c6-ee49868ad001 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:18.365797 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921e02e-d1ce-4b7a-b0c6-ee49868ad001 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:18.366219 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921e02e-d1ce-4b7a-b0c6-ee49868ad001 None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:18.367228 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6921e02e-d1ce-4b7a-b0c6-ee49868ad001 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:18.367712 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6921e02e-d1ce-4b7a-b0c6-ee49868ad001 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:18.368562 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb15760-0de6-4939-a676-37a322b0d4fc None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:18.369436 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6921e02e-d1ce-4b7a-b0c6-ee49868ad001 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:18.372693 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:18.383483 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 202/414] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:15:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:15:18.389119 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6921e02e-d1ce-4b7a-b0c6-ee49868ad001 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:18.485678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb15760-0de6-4939-a676-37a322b0d4fc None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:18.529489 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f8b65-39f4-4131-a8a5-7b8a2515a153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:18.530930 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f8b65-39f4-4131-a8a5-7b8a2515a153 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:18.531803 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f8b65-39f4-4131-a8a5-7b8a2515a153 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:18.533125 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f8b65-39f4-4131-a8a5-7b8a2515a153 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:18.546165 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1f8b65-39f4-4131-a8a5-7b8a2515a153 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:18.556247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-6e1f8b65-39f4-4131-a8a5-7b8a2515a153 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 07 15:15:18.556247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:18.556247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:18.556247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:18.556247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:18.556247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:18.556247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:18.556247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:18.556247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:18.556247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:18.556247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:18.556247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:18.556247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:18.556247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:18.556247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:18.556247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:18.557138 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:18.557138 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:18.557138 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:18.557138 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:18.557138 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:18.557138 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:18.557138 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:18.557138 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:18.557138 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:18.557138 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:18.557138 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:18.557138 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:18.557138 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:18.557138 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:18.557138 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:18.557138 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:18.558018 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:18.558018 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:18.558018 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:18.558018 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:18.558018 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:18.558018 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:18.558018 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:18.558018 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:18.558018 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:18.558018 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:18.558018 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 07 15:15:18.558018 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:18.563831 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 213/415] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:15:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:15:18.569010 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f8b65-39f4-4131-a8a5-7b8a2515a153 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:18.570453 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6921e02e-d1ce-4b7a-b0c6-ee49868ad001 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:18.614104 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e705abb-c094-42f2-b5d9-8136890295e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:18.614726 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e705abb-c094-42f2-b5d9-8136890295e3 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:18.615014 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e705abb-c094-42f2-b5d9-8136890295e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:18.615290 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e705abb-c094-42f2-b5d9-8136890295e3 None admin] PATH_INFO: `/v3/groups/8ccaec72b7db46078702143bc510dafa` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:18.616565 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e705abb-c094-42f2-b5d9-8136890295e3 None admin] RBAC: Authorizing `identity:get_group(group_id=8ccaec72b7db46078702143bc510dafa)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:18.618012 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e705abb-c094-42f2-b5d9-8136890295e3 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:18.620321 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 203/416] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Jul 7 15:15:18 2022] GET /identity/v3/groups/8ccaec72b7db46078702143bc510dafa => generated 229 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:15:18.628129 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:18.628857 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e705abb-c094-42f2-b5d9-8136890295e3 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:18.687853 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f8b65-39f4-4131-a8a5-7b8a2515a153 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:18.727009 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b44cc7-c765-44a0-9317-15e3ea1454f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:18.727869 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b44cc7-c765-44a0-9317-15e3ea1454f3 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:18.728253 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b44cc7-c765-44a0-9317-15e3ea1454f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:18.728618 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b44cc7-c765-44a0-9317-15e3ea1454f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:18.729618 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b44cc7-c765-44a0-9317-15e3ea1454f3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:18.730024 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b44cc7-c765-44a0-9317-15e3ea1454f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:18.732184 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b44cc7-c765-44a0-9317-15e3ea1454f3 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:18.747641 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 214/417] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Jul 7 15:15:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:15:18.749475 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b44cc7-c765-44a0-9317-15e3ea1454f3 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:18.766887 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:18.866325 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e705abb-c094-42f2-b5d9-8136890295e3 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:18.907126 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c7b41-d354-40e1-b055-476e475d21e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:18.908359 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c7b41-d354-40e1-b055-476e475d21e0 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:18.908841 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c7b41-d354-40e1-b055-476e475d21e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:18.909322 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c7b41-d354-40e1-b055-476e475d21e0 None admin] PATH_INFO: `/v3/projects/28133f9a98be4af88ed357f76f5a4a2e` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:18.910841 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c7b41-d354-40e1-b055-476e475d21e0 None admin] RBAC: Authorizing `identity:get_project(project_id=28133f9a98be4af88ed357f76f5a4a2e)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:18.912524 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c7b41-d354-40e1-b055-476e475d21e0 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:18.914791 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 204/418] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:15:18 2022] GET /identity/v3/projects/28133f9a98be4af88ed357f76f5a4a2e => generated 344 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:18.920047 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c7b41-d354-40e1-b055-476e475d21e0 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:18.921992 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b44cc7-c765-44a0-9317-15e3ea1454f3 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:18.942563 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:19.021083 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469dea01-aac2-4f20-9d03-100dc8680c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:19.021764 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469dea01-aac2-4f20-9d03-100dc8680c95 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:19.022051 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469dea01-aac2-4f20-9d03-100dc8680c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:19.022330 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469dea01-aac2-4f20-9d03-100dc8680c95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:19.049472 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c7b41-d354-40e1-b055-476e475d21e0 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:19.099503 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c12b8a-f46a-4fa2-91a9-71d9800a339e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:19.100885 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c12b8a-f46a-4fa2-91a9-71d9800a339e None admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:19.101359 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c12b8a-f46a-4fa2-91a9-71d9800a339e None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:19.101827 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c12b8a-f46a-4fa2-91a9-71d9800a339e None admin] PATH_INFO: `/v3/projects/28133f9a98be4af88ed357f76f5a4a2e/groups/8ccaec72b7db46078702143bc510dafa/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:19.104049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c12b8a-f46a-4fa2-91a9-71d9800a339e None admin] RBAC: Authorizing `identity:create_grant(project_id=28133f9a98be4af88ed357f76f5a4a2e, group_id=8ccaec72b7db46078702143bc510dafa, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:19.105686 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c12b8a-f46a-4fa2-91a9-71d9800a339e None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:19.108208 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:19.143670 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469dea01-aac2-4f20-9d03-100dc8680c95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:19.145071 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469dea01-aac2-4f20-9d03-100dc8680c95 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:19.205846 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 205/419] 10.222.0.25 () {58 vars in 1442 bytes} [Thu Jul 7 15:15:18 2022] PUT /identity/v3/projects/28133f9a98be4af88ed357f76f5a4a2e/groups/8ccaec72b7db46078702143bc510dafa/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:15:19.210818 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebe481-8b9e-4cce-86dc-4fe6082ddb77 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:19.215009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebe481-8b9e-4cce-86dc-4fe6082ddb77 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:19.215009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebe481-8b9e-4cce-86dc-4fe6082ddb77 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:19.216757 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 206/420] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:19 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:19.223113 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1edc2f-c330-4640-8810-816cd8144799 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:19.223686 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1edc2f-c330-4640-8810-816cd8144799 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:19.224500 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1edc2f-c330-4640-8810-816cd8144799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:19.252861 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 215/421] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:15:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 15:15:19.258956 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469dea01-aac2-4f20-9d03-100dc8680c95 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:19.267990 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-9b1edc2f-c330-4640-8810-816cd8144799 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:19.269404 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:19.270558 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-9b1edc2f-c330-4640-8810-816cd8144799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:19.357914 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 207/422] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:19.396682 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469dea01-aac2-4f20-9d03-100dc8680c95 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:19.402986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1edc2f-c330-4640-8810-816cd8144799 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:19.435969 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e076c-a88d-45a4-9fdd-2d4e69bd3cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:19.437070 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e076c-a88d-45a4-9fdd-2d4e69bd3cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:19.437351 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e076c-a88d-45a4-9fdd-2d4e69bd3cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:19.437622 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e076c-a88d-45a4-9fdd-2d4e69bd3cf7 None admin] PATH_INFO: `/v3/projects/db427256e2ec4ec084f762a615d0f187` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:19.438853 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4e076c-a88d-45a4-9fdd-2d4e69bd3cf7 None admin] RBAC: Authorizing `identity:get_project(project_id=db427256e2ec4ec084f762a615d0f187)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:19.440726 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4e076c-a88d-45a4-9fdd-2d4e69bd3cf7 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:19.447346 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 216/423] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:15:19 2022] GET /identity/v3/projects/db427256e2ec4ec084f762a615d0f187 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:19.448318 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e076c-a88d-45a4-9fdd-2d4e69bd3cf7 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:19.539061 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:19.561059 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e076c-a88d-45a4-9fdd-2d4e69bd3cf7 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:19.598472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07acd61-4736-46a6-b1c2-50480cc96074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:19.599325 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07acd61-4736-46a6-b1c2-50480cc96074 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:19.599628 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07acd61-4736-46a6-b1c2-50480cc96074 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:19.599903 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07acd61-4736-46a6-b1c2-50480cc96074 None admin] PATH_INFO: `/v3/groups/dd675b91e3ef45dc8650e456e0529fd7` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:19.601265 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07acd61-4736-46a6-b1c2-50480cc96074 None admin] RBAC: Authorizing `identity:get_group(group_id=dd675b91e3ef45dc8650e456e0529fd7)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:19.602514 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07acd61-4736-46a6-b1c2-50480cc96074 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:19.604154 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 217/424] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Jul 7 15:15:19 2022] GET /identity/v3/groups/dd675b91e3ef45dc8650e456e0529fd7 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:19.610184 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07acd61-4736-46a6-b1c2-50480cc96074 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:19.615630 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1edc2f-c330-4640-8810-816cd8144799 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:19.650480 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:19.675849 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b0d34-6b67-42d1-99c5-6541534316d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:19.677053 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b0d34-6b67-42d1-99c5-6541534316d0 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:19.677579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b0d34-6b67-42d1-99c5-6541534316d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:19.678043 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b0d34-6b67-42d1-99c5-6541534316d0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:19.700461 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b0d34-6b67-42d1-99c5-6541534316d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:19.715589 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-412b0d34-6b67-42d1-99c5-6541534316d0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 15:15:19.715589 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:19.715589 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:19.715589 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:19.715589 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:19.715589 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:19.715589 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:19.715589 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:19.715589 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:19.715589 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:19.715589 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:19.715589 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:19.715589 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:19.715589 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:19.715589 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:19.715589 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:19.716534 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:19.716534 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:19.716534 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:19.716534 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:19.716534 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:19.716534 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:19.716534 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:19.716534 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:19.716534 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:19.716534 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:19.716534 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:19.716534 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:19.716534 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:19.716534 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:19.716534 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:19.716534 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:19.717845 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:19.717845 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:19.717845 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:19.717845 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:19.717845 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:19.717845 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:19.717845 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:19.717845 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:19.717845 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:19.717845 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:19.717845 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 15:15:19.717845 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:19.718462 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07acd61-4736-46a6-b1c2-50480cc96074 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:19.721674 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 208/425] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Jul 7 15:15:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:15:19.726434 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b0d34-6b67-42d1-99c5-6541534316d0 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:19.754755 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd9b53f-922f-4f93-b84a-c067ead0a0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:19.755440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9b53f-922f-4f93-b84a-c067ead0a0c6 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:19.756021 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9b53f-922f-4f93-b84a-c067ead0a0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:19.756127 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9b53f-922f-4f93-b84a-c067ead0a0c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:19.756888 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd9b53f-922f-4f93-b84a-c067ead0a0c6 None admin] RBAC: Adding query filter params (group.id=dd675b91e3ef45dc8650e456e0529fd7, role.id=d43d0d189e254500875f5f86e6abd178, scope.project.id=db427256e2ec4ec084f762a615d0f187) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:19.757199 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd9b53f-922f-4f93-b84a-c067ead0a0c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:19.758731 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd9b53f-922f-4f93-b84a-c067ead0a0c6 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:19.765631 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 218/426] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Jul 7 15:15:19 2022] GET /identity/v3/role_assignments?group.id=dd675b91e3ef45dc8650e456e0529fd7&scope.project.id=db427256e2ec4ec084f762a615d0f187&role.id=d43d0d189e254500875f5f86e6abd178 => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:19.851023 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b0d34-6b67-42d1-99c5-6541534316d0 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:19.892101 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f72dc0-2a59-4ff9-8141-bf411c7c3d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:19.893669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f72dc0-2a59-4ff9-8141-bf411c7c3d67 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:19.894170 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f72dc0-2a59-4ff9-8141-bf411c7c3d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:19.894735 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f72dc0-2a59-4ff9-8141-bf411c7c3d67 None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:19.895735 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f72dc0-2a59-4ff9-8141-bf411c7c3d67 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:19.896247 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f72dc0-2a59-4ff9-8141-bf411c7c3d67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:19.897841 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f72dc0-2a59-4ff9-8141-bf411c7c3d67 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:19.908611 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 209/427] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:15:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:15:19.915688 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd9b53f-922f-4f93-b84a-c067ead0a0c6 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:19.935974 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:20.072832 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd9b53f-922f-4f93-b84a-c067ead0a0c6 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:20.111910 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2fe33a-6602-4fbd-b1ac-5bc44955cf2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:20.112624 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2fe33a-6602-4fbd-b1ac-5bc44955cf2b None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:20.112904 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2fe33a-6602-4fbd-b1ac-5bc44955cf2b None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:20.113227 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2fe33a-6602-4fbd-b1ac-5bc44955cf2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:20.151101 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd0c2b4-010c-45c5-af47-22d4eca6b997 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:20.151432 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd0c2b4-010c-45c5-af47-22d4eca6b997 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:20.151732 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd0c2b4-010c-45c5-af47-22d4eca6b997 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:20.153326 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 210/428] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:20.159610 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee03e0c-f2e4-4cc1-842f-86797207e0c9 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:20.159865 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee03e0c-f2e4-4cc1-842f-86797207e0c9 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:20.160144 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee03e0c-f2e4-4cc1-842f-86797207e0c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:20.185139 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-cee03e0c-f2e4-4cc1-842f-86797207e0c9 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:20.186326 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-cee03e0c-f2e4-4cc1-842f-86797207e0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:20.187550 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2fe33a-6602-4fbd-b1ac-5bc44955cf2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:20.189224 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2fe33a-6602-4fbd-b1ac-5bc44955cf2b None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:20.200016 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:20.241307 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:20.275010 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 211/429] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 15:15:20.303401 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee03e0c-f2e4-4cc1-842f-86797207e0c9 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:20.320834 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 219/430] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:15:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 15:15:20.326226 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2fe33a-6602-4fbd-b1ac-5bc44955cf2b None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:20.436784 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee03e0c-f2e4-4cc1-842f-86797207e0c9 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:20.447735 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2fe33a-6602-4fbd-b1ac-5bc44955cf2b None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:20.459706 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:20.488261 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463152f-5dc2-47c4-b35c-442d34917a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:20.489291 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463152f-5dc2-47c4-b35c-442d34917a94 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:20.489425 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54879651-363f-4f9a-8be1-df0ce5cbb747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:20.489632 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463152f-5dc2-47c4-b35c-442d34917a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:20.490049 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463152f-5dc2-47c4-b35c-442d34917a94 None admin] PATH_INFO: `/v3/projects/db427256e2ec4ec084f762a615d0f187` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:20.490157 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54879651-363f-4f9a-8be1-df0ce5cbb747 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:20.490349 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54879651-363f-4f9a-8be1-df0ce5cbb747 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:20.490625 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54879651-363f-4f9a-8be1-df0ce5cbb747 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:20.491872 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2463152f-5dc2-47c4-b35c-442d34917a94 None admin] RBAC: Authorizing `identity:get_project(project_id=db427256e2ec4ec084f762a615d0f187)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:20.493349 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2463152f-5dc2-47c4-b35c-442d34917a94 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:20.501761 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 220/431] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:15:20 2022] GET /identity/v3/projects/db427256e2ec4ec084f762a615d0f187 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:15:20.502662 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463152f-5dc2-47c4-b35c-442d34917a94 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:20.505577 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54879651-363f-4f9a-8be1-df0ce5cbb747 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:20.514980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-54879651-363f-4f9a-8be1-df0ce5cbb747 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 07 15:15:20.514980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:20.514980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:20.514980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:20.514980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:20.514980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:20.514980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:20.514980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:20.514980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:20.514980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:20.514980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:20.514980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:20.514980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:20.514980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:20.514980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:20.514980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:20.516481 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:20.516481 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:20.516481 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:20.516481 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:20.516481 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:20.516481 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:20.516481 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:20.516481 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:20.516481 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:20.516481 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:20.516481 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:20.516481 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:20.516481 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:20.516481 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:20.516481 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:20.516481 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:20.517320 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:20.517320 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:20.517320 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:20.517320 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:20.517320 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:20.517320 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:20.517320 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:20.517320 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:20.517320 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:20.517320 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:20.517320 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 07 15:15:20.517320 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:20.518774 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 212/432] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:15:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:15:20.523764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54879651-363f-4f9a-8be1-df0ce5cbb747 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:20.576678 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:20.627083 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463152f-5dc2-47c4-b35c-442d34917a94 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:20.641951 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54879651-363f-4f9a-8be1-df0ce5cbb747 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:20.664964 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c32ba1-3be5-4c07-99bd-72d6f325c095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:20.665826 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c32ba1-3be5-4c07-99bd-72d6f325c095 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:20.666107 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c32ba1-3be5-4c07-99bd-72d6f325c095 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:20.666382 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c32ba1-3be5-4c07-99bd-72d6f325c095 None admin] PATH_INFO: `/v3/groups/dd675b91e3ef45dc8650e456e0529fd7` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:20.667592 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c32ba1-3be5-4c07-99bd-72d6f325c095 None admin] RBAC: Authorizing `identity:get_group(group_id=dd675b91e3ef45dc8650e456e0529fd7)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:20.669061 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c32ba1-3be5-4c07-99bd-72d6f325c095 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:20.670975 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 221/433] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Jul 7 15:15:20 2022] GET /identity/v3/groups/dd675b91e3ef45dc8650e456e0529fd7 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:20.677771 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c32ba1-3be5-4c07-99bd-72d6f325c095 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:20.684280 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616722fe-82e6-4f67-846a-16acbcf63762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:20.684952 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616722fe-82e6-4f67-846a-16acbcf63762 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:20.685233 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616722fe-82e6-4f67-846a-16acbcf63762 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:20.685508 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616722fe-82e6-4f67-846a-16acbcf63762 None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:20.686264 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616722fe-82e6-4f67-846a-16acbcf63762 None admin] RBAC: Adding query filter params (name=member) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:20.686571 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616722fe-82e6-4f67-846a-16acbcf63762 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:20.689597 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616722fe-82e6-4f67-846a-16acbcf63762 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:20.698859 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 213/434] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Jul 7 15:15:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:20.708446 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616722fe-82e6-4f67-846a-16acbcf63762 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:20.738089 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:20.816546 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c32ba1-3be5-4c07-99bd-72d6f325c095 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:20.842327 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:20.846967 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616722fe-82e6-4f67-846a-16acbcf63762 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:20.868237 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d9e68b-bf99-411e-b3cb-b06b14f16e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:20.869349 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9e68b-bf99-411e-b3cb-b06b14f16e09 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:20.870071 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9e68b-bf99-411e-b3cb-b06b14f16e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:20.870495 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9e68b-bf99-411e-b3cb-b06b14f16e09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:20.871661 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d9e68b-bf99-411e-b3cb-b06b14f16e09 None admin] RBAC: Adding query filter params (group.id=dd675b91e3ef45dc8650e456e0529fd7, role.id=c82203e4734346719cafb16230f45942, scope.project.id=db427256e2ec4ec084f762a615d0f187) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:20.872119 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d9e68b-bf99-411e-b3cb-b06b14f16e09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:20.875983 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d9e68b-bf99-411e-b3cb-b06b14f16e09 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:20.885460 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 222/435] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Jul 7 15:15:20 2022] GET /identity/v3/role_assignments?group.id=dd675b91e3ef45dc8650e456e0529fd7&scope.project.id=db427256e2ec4ec084f762a615d0f187&role.id=c82203e4734346719cafb16230f45942 => generated 611 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:20.892379 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee5c326-248e-41d4-8ae2-93fc724bc520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:20.893627 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5c326-248e-41d4-8ae2-93fc724bc520 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:20.894075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5c326-248e-41d4-8ae2-93fc724bc520 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:20.894495 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5c326-248e-41d4-8ae2-93fc724bc520 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:20.977352 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee5c326-248e-41d4-8ae2-93fc724bc520 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:20.978835 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee5c326-248e-41d4-8ae2-93fc724bc520 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:20.989776 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc83e4c-e916-4319-a65f-d9db79096e17 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:20.990216 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc83e4c-e916-4319-a65f-d9db79096e17 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:20.990600 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc83e4c-e916-4319-a65f-d9db79096e17 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:20.996213 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 223/436] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:15:20.998141 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9ce95-eab8-44ba-a15a-e29a28852105 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:20.998540 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9ce95-eab8-44ba-a15a-e29a28852105 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:20.998921 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9ce95-eab8-44ba-a15a-e29a28852105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:21.020319 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-84b9ce95-eab8-44ba-a15a-e29a28852105 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:21.021735 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-84b9ce95-eab8-44ba-a15a-e29a28852105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:21.086662 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:21.095280 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 214/437] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:15:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 15:15:21.101582 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee5c326-248e-41d4-8ae2-93fc724bc520 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:21.115160 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 224/438] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:21.138589 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9ce95-eab8-44ba-a15a-e29a28852105 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:21.197263 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:21.230694 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee5c326-248e-41d4-8ae2-93fc724bc520 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:21.275977 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9ce95-eab8-44ba-a15a-e29a28852105 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:21.277665 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c793b34-e5d2-4407-8d78-d1a023c61310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:21.278544 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c793b34-e5d2-4407-8d78-d1a023c61310 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:21.279687 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c793b34-e5d2-4407-8d78-d1a023c61310 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:21.279972 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c793b34-e5d2-4407-8d78-d1a023c61310 None admin] PATH_INFO: `/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:21.281204 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c793b34-e5d2-4407-8d78-d1a023c61310 None admin] RBAC: Authorizing `identity:get_project(project_id=a6fde692f8db4d7eb4858942b0b6e5bf)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:21.282665 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c793b34-e5d2-4407-8d78-d1a023c61310 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:21.285056 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 215/439] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:15:21 2022] GET /identity/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:21.290165 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c793b34-e5d2-4407-8d78-d1a023c61310 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:21.318781 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e76c5e9-dcab-4590-9618-0f9b304f7a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:21.320699 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76c5e9-dcab-4590-9618-0f9b304f7a60 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:21.320699 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76c5e9-dcab-4590-9618-0f9b304f7a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:21.320699 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76c5e9-dcab-4590-9618-0f9b304f7a60 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:21.332865 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e76c5e9-dcab-4590-9618-0f9b304f7a60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:21.343167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-4e76c5e9-dcab-4590-9618-0f9b304f7a60 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 15:15:21.343167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:21.343167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:21.343167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:21.343167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:21.343167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:21.343167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:21.343167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:21.343167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:21.343167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:21.343167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:21.343167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:21.343167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:21.343167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:21.343167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:21.343167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:21.344955 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:21.344955 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:21.344955 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:21.344955 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:21.344955 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:21.344955 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:21.344955 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:21.344955 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:21.344955 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:21.344955 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:21.344955 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:21.344955 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:21.344955 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:21.344955 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:21.344955 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:21.344955 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:21.345784 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:21.345784 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:21.345784 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:21.345784 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:21.345784 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:21.345784 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:21.345784 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:21.345784 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:21.345784 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:21.345784 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:21.345784 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 15:15:21.345784 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:21.347445 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 225/440] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Jul 7 15:15:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:15:21.354229 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e76c5e9-dcab-4590-9618-0f9b304f7a60 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:21.383737 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:21.427869 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c793b34-e5d2-4407-8d78-d1a023c61310 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:21.458893 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:21.462594 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6082d105-a6af-465d-89e4-8f5922ffc3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:21.462727 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e76c5e9-dcab-4590-9618-0f9b304f7a60 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:21.463453 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082d105-a6af-465d-89e4-8f5922ffc3d3 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:21.463781 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082d105-a6af-465d-89e4-8f5922ffc3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:21.464095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082d105-a6af-465d-89e4-8f5922ffc3d3 None admin] PATH_INFO: `/v3/groups/dd675b91e3ef45dc8650e456e0529fd7` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:21.465528 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6082d105-a6af-465d-89e4-8f5922ffc3d3 None admin] RBAC: Authorizing `identity:get_group(group_id=dd675b91e3ef45dc8650e456e0529fd7)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:21.466709 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6082d105-a6af-465d-89e4-8f5922ffc3d3 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:21.468518 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 216/441] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Jul 7 15:15:21 2022] GET /identity/v3/groups/dd675b91e3ef45dc8650e456e0529fd7 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:15:21.475002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6082d105-a6af-465d-89e4-8f5922ffc3d3 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:21.500906 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63beba1-a3e3-4edb-9454-9c7c4b3bf688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:21.501497 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63beba1-a3e3-4edb-9454-9c7c4b3bf688 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:21.501777 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63beba1-a3e3-4edb-9454-9c7c4b3bf688 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:21.502053 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63beba1-a3e3-4edb-9454-9c7c4b3bf688 None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:21.502802 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63beba1-a3e3-4edb-9454-9c7c4b3bf688 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:21.503107 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63beba1-a3e3-4edb-9454-9c7c4b3bf688 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:21.504522 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63beba1-a3e3-4edb-9454-9c7c4b3bf688 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:21.513423 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 226/442] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:15:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:15:21.518876 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63beba1-a3e3-4edb-9454-9c7c4b3bf688 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:21.579315 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6082d105-a6af-465d-89e4-8f5922ffc3d3 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:21.615078 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe6db5-2511-4d72-ba56-4dfbd48659e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:21.616288 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe6db5-2511-4d72-ba56-4dfbd48659e5 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:21.616574 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe6db5-2511-4d72-ba56-4dfbd48659e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:21.616988 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe6db5-2511-4d72-ba56-4dfbd48659e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:21.617959 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efe6db5-2511-4d72-ba56-4dfbd48659e5 None admin] RBAC: Adding query filter params (group.id=dd675b91e3ef45dc8650e456e0529fd7, role.id=d43d0d189e254500875f5f86e6abd178, scope.project.id=a6fde692f8db4d7eb4858942b0b6e5bf) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:21.618526 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efe6db5-2511-4d72-ba56-4dfbd48659e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:21.620729 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efe6db5-2511-4d72-ba56-4dfbd48659e5 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:21.623499 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:21.628663 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63beba1-a3e3-4edb-9454-9c7c4b3bf688 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:21.634149 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 217/443] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Jul 7 15:15:21 2022] GET /identity/v3/role_assignments?group.id=dd675b91e3ef45dc8650e456e0529fd7&scope.project.id=a6fde692f8db4d7eb4858942b0b6e5bf&role.id=d43d0d189e254500875f5f86e6abd178 => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:21.666705 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c1e68-444f-4436-868a-f0d17f7e4ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:21.667363 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c1e68-444f-4436-868a-f0d17f7e4ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:21.667699 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c1e68-444f-4436-868a-f0d17f7e4ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:21.667975 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c1e68-444f-4436-868a-f0d17f7e4ad6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:21.699890 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:21.744091 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142c1e68-444f-4436-868a-f0d17f7e4ad6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:21.745578 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142c1e68-444f-4436-868a-f0d17f7e4ad6 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:21.755864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181586b-b2a6-467e-9582-279fac839667 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:21.756109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181586b-b2a6-467e-9582-279fac839667 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:21.756444 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181586b-b2a6-467e-9582-279fac839667 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:21.757929 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 218/444] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:21.764284 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4921d8-bbe1-40c9-bc0b-0217cdfe4ac1 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:21.764569 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4921d8-bbe1-40c9-bc0b-0217cdfe4ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:21.764844 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4921d8-bbe1-40c9-bc0b-0217cdfe4ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:21.786097 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-3f4921d8-bbe1-40c9-bc0b-0217cdfe4ac1 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:21.787309 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-3f4921d8-bbe1-40c9-bc0b-0217cdfe4ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:21.861235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 227/445] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:15:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 07 15:15:21.866802 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c1e68-444f-4436-868a-f0d17f7e4ad6 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:21.876237 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 219/446] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 15:15:21.900096 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4921d8-bbe1-40c9-bc0b-0217cdfe4ac1 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:21.933388 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:21.985732 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c1e68-444f-4436-868a-f0d17f7e4ad6 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:22.017954 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4921d8-bbe1-40c9-bc0b-0217cdfe4ac1 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:22.021581 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c41744-6e07-4bf5-b5ea-4d0212e5d4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:22.022438 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c41744-6e07-4bf5-b5ea-4d0212e5d4b8 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:22.022719 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c41744-6e07-4bf5-b5ea-4d0212e5d4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:22.023001 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c41744-6e07-4bf5-b5ea-4d0212e5d4b8 None admin] PATH_INFO: `/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:22.024195 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c41744-6e07-4bf5-b5ea-4d0212e5d4b8 None admin] RBAC: Authorizing `identity:get_project(project_id=a6fde692f8db4d7eb4858942b0b6e5bf)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:22.025625 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c41744-6e07-4bf5-b5ea-4d0212e5d4b8 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:22.027408 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 228/447] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:15:21 2022] GET /identity/v3/projects/a6fde692f8db4d7eb4858942b0b6e5bf => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:15:22.033305 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c41744-6e07-4bf5-b5ea-4d0212e5d4b8 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:22.043940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:22.051448 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b0bdb-122a-4ac4-85eb-0d3776f5239f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:22.052243 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b0bdb-122a-4ac4-85eb-0d3776f5239f None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:22.055435 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b0bdb-122a-4ac4-85eb-0d3776f5239f None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:22.055435 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b0bdb-122a-4ac4-85eb-0d3776f5239f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:22.063089 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5b0bdb-122a-4ac4-85eb-0d3776f5239f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:22.070813 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-0b5b0bdb-122a-4ac4-85eb-0d3776f5239f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:15:22.070813 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:22.070813 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:22.070813 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:22.070813 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:22.070813 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:22.070813 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:22.070813 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:22.070813 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:22.070813 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:22.070813 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:22.070813 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:22.070813 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:22.070813 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:22.070813 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:22.070813 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:22.071834 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:22.071834 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:22.071834 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:22.071834 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:22.071834 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:22.071834 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:22.071834 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:22.071834 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:22.071834 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:22.071834 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:22.071834 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:22.071834 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:22.071834 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:22.071834 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:22.071834 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:22.071834 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:22.072647 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:22.072647 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:22.072647 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:22.072647 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:22.072647 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:22.072647 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:22.072647 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:22.072647 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:22.072647 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:22.072647 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:22.072647 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:15:22.072647 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:22.075705 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 220/448] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Jul 7 15:15:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:22.077655 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b0bdb-122a-4ac4-85eb-0d3776f5239f None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:22.131249 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c41744-6e07-4bf5-b5ea-4d0212e5d4b8 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:22.157593 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:22.165804 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873b8cd-0894-4699-a08c-789e2d8ac6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:22.169880 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873b8cd-0894-4699-a08c-789e2d8ac6c9 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:22.169880 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873b8cd-0894-4699-a08c-789e2d8ac6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:22.169880 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873b8cd-0894-4699-a08c-789e2d8ac6c9 None admin] PATH_INFO: `/v3/groups/dd675b91e3ef45dc8650e456e0529fd7` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:22.169880 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0873b8cd-0894-4699-a08c-789e2d8ac6c9 None admin] RBAC: Authorizing `identity:get_group(group_id=dd675b91e3ef45dc8650e456e0529fd7)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:22.169880 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0873b8cd-0894-4699-a08c-789e2d8ac6c9 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:22.171410 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 229/449] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Jul 7 15:15:22 2022] GET /identity/v3/groups/dd675b91e3ef45dc8650e456e0529fd7 => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:15:22.177100 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873b8cd-0894-4699-a08c-789e2d8ac6c9 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:22.182365 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b0bdb-122a-4ac4-85eb-0d3776f5239f None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:22.216296 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d651b-d65b-4cd5-aa29-cfd163ff597f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:22.216985 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d651b-d65b-4cd5-aa29-cfd163ff597f None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:22.217267 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d651b-d65b-4cd5-aa29-cfd163ff597f None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:22.217543 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9d651b-d65b-4cd5-aa29-cfd163ff597f None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:22.218292 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9d651b-d65b-4cd5-aa29-cfd163ff597f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:22.218601 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9d651b-d65b-4cd5-aa29-cfd163ff597f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:22.220022 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9d651b-d65b-4cd5-aa29-cfd163ff597f None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:22.228776 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 221/450] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:15:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:15:22.234085 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d651b-d65b-4cd5-aa29-cfd163ff597f None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:22.280042 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873b8cd-0894-4699-a08c-789e2d8ac6c9 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:22.283748 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:22.312375 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf20adb-9920-4616-a0de-33bfb46defef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:22.313967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf20adb-9920-4616-a0de-33bfb46defef None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:22.314273 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf20adb-9920-4616-a0de-33bfb46defef None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:22.314547 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf20adb-9920-4616-a0de-33bfb46defef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:22.315374 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf20adb-9920-4616-a0de-33bfb46defef None admin] RBAC: Adding query filter params (group.id=dd675b91e3ef45dc8650e456e0529fd7, role.id=c82203e4734346719cafb16230f45942, scope.project.id=a6fde692f8db4d7eb4858942b0b6e5bf) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:22.315715 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf20adb-9920-4616-a0de-33bfb46defef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:22.317136 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf20adb-9920-4616-a0de-33bfb46defef None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:22.322892 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 230/451] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Jul 7 15:15:22 2022] GET /identity/v3/role_assignments?group.id=dd675b91e3ef45dc8650e456e0529fd7&scope.project.id=a6fde692f8db4d7eb4858942b0b6e5bf&role.id=c82203e4734346719cafb16230f45942 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:15:22.334837 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9d651b-d65b-4cd5-aa29-cfd163ff597f None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:22.370805 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f6bd72-bcff-4994-9dda-8e4a21086c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:22.371463 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6bd72-bcff-4994-9dda-8e4a21086c77 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:22.371772 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6bd72-bcff-4994-9dda-8e4a21086c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:22.372046 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6bd72-bcff-4994-9dda-8e4a21086c77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:22.443695 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f6bd72-bcff-4994-9dda-8e4a21086c77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:22.445157 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f6bd72-bcff-4994-9dda-8e4a21086c77 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:22.499945 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:22.551751 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 222/452] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:15:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 07 15:15:22.557327 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf20adb-9920-4616-a0de-33bfb46defef None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:22.616988 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:22.657884 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf20adb-9920-4616-a0de-33bfb46defef None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:22.693172 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18932734-5f80-4f99-a987-fe60c082453d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:22.694026 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18932734-5f80-4f99-a987-fe60c082453d None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:22.694310 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18932734-5f80-4f99-a987-fe60c082453d None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:22.694585 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18932734-5f80-4f99-a987-fe60c082453d None admin] PATH_INFO: `/v3/projects/28133f9a98be4af88ed357f76f5a4a2e` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:22.695774 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18932734-5f80-4f99-a987-fe60c082453d None admin] RBAC: Authorizing `identity:get_project(project_id=28133f9a98be4af88ed357f76f5a4a2e)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:22.697214 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18932734-5f80-4f99-a987-fe60c082453d None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:22.698884 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 231/453] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:15:22 2022] GET /identity/v3/projects/28133f9a98be4af88ed357f76f5a4a2e => generated 344 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:15:22.704362 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f6bd72-bcff-4994-9dda-8e4a21086c77 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:22.799822 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f6bd72-bcff-4994-9dda-8e4a21086c77 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:22.833376 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fafa71-f331-457d-a527-27aba6f99f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:22.834596 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fafa71-f331-457d-a527-27aba6f99f08 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:22.834885 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fafa71-f331-457d-a527-27aba6f99f08 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:22.835166 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fafa71-f331-457d-a527-27aba6f99f08 None admin] PATH_INFO: `/v3/groups/8ccaec72b7db46078702143bc510dafa` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:22.836377 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fafa71-f331-457d-a527-27aba6f99f08 None admin] RBAC: Authorizing `identity:get_group(group_id=8ccaec72b7db46078702143bc510dafa)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:22.837832 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fafa71-f331-457d-a527-27aba6f99f08 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:22.839445 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 223/454] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Jul 7 15:15:22 2022] GET /identity/v3/groups/8ccaec72b7db46078702143bc510dafa => generated 229 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:22.845659 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18932734-5f80-4f99-a987-fe60c082453d None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:22.944178 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18932734-5f80-4f99-a987-fe60c082453d None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:22.969821 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:22.977469 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32a3fa-ad9f-462c-9bf6-35705cd1c8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:22.978114 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32a3fa-ad9f-462c-9bf6-35705cd1c8a6 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:22.978396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32a3fa-ad9f-462c-9bf6-35705cd1c8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:22.978670 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32a3fa-ad9f-462c-9bf6-35705cd1c8a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:22.979493 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32a3fa-ad9f-462c-9bf6-35705cd1c8a6 None admin] RBAC: Adding query filter params (group.id=8ccaec72b7db46078702143bc510dafa, role.id=0e7793fbf3064037984da651879e3a94, scope.project.id=28133f9a98be4af88ed357f76f5a4a2e) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:22.979832 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32a3fa-ad9f-462c-9bf6-35705cd1c8a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:22.982857 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32a3fa-ad9f-462c-9bf6-35705cd1c8a6 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:22.987737 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 232/455] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Jul 7 15:15:22 2022] GET /identity/v3/role_assignments?group.id=8ccaec72b7db46078702143bc510dafa&scope.project.id=28133f9a98be4af88ed357f76f5a4a2e&role.id=0e7793fbf3064037984da651879e3a94 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:25.632572 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e176772-8132-442e-b7bf-2b2445f366de None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:25.633079 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e176772-8132-442e-b7bf-2b2445f366de None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:25.633156 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e176772-8132-442e-b7bf-2b2445f366de None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:25.634736 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 224/456] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:25.640185 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e1056-bc86-42d4-be6e-08dae9fa917c None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:25.641054 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e1056-bc86-42d4-be6e-08dae9fa917c None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:25.641533 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e1056-bc86-42d4-be6e-08dae9fa917c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:25.658447 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-0e2e1056-bc86-42d4-be6e-08dae9fa917c None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:25.660068 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-0e2e1056-bc86-42d4-be6e-08dae9fa917c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:25.721134 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa12c6e-70b4-4a11-975c-92ec1e33521f None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:25.721746 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa12c6e-70b4-4a11-975c-92ec1e33521f None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:25.722225 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa12c6e-70b4-4a11-975c-92ec1e33521f None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:25.724320 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 225/457] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:15:25.734224 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7288faa-1561-4a96-a043-71e38a261cd9 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:25.735342 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7288faa-1561-4a96-a043-71e38a261cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:25.735842 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7288faa-1561-4a96-a043-71e38a261cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:25.753418 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-b7288faa-1561-4a96-a043-71e38a261cd9 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:25.754935 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-b7288faa-1561-4a96-a043-71e38a261cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:25.775396 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:25.781095 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 233/458] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:25.838659 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29560288-f118-4bdb-bdf0-299a7329a1ee None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:25.838978 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29560288-f118-4bdb-bdf0-299a7329a1ee None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:25.839228 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29560288-f118-4bdb-bdf0-299a7329a1ee None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:25.841554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 234/459] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:15:25.846958 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335c056-bb05-4679-9a0a-5dd0869f31a9 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:25.848017 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335c056-bb05-4679-9a0a-5dd0869f31a9 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:25.848607 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335c056-bb05-4679-9a0a-5dd0869f31a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:25.856993 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 226/460] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 15:15:25.857855 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7288faa-1561-4a96-a043-71e38a261cd9 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:25.875971 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-4335c056-bb05-4679-9a0a-5dd0869f31a9 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:25.877163 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-4335c056-bb05-4679-9a0a-5dd0869f31a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:25.983015 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 235/461] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 15:15:25.983015 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4335c056-bb05-4679-9a0a-5dd0869f31a9 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:25.993713 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7288faa-1561-4a96-a043-71e38a261cd9 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:26.039873 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b6d1a-5a77-44b6-a2af-f8c7b605972f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:26.040731 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b6d1a-5a77-44b6-a2af-f8c7b605972f None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:26.041020 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b6d1a-5a77-44b6-a2af-f8c7b605972f None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:26.041302 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b6d1a-5a77-44b6-a2af-f8c7b605972f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:26.051328 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:26.052347 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b6d1a-5a77-44b6-a2af-f8c7b605972f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:26.053807 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b6d1a-5a77-44b6-a2af-f8c7b605972f None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:26.063762 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:26.073184 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-0b4b6d1a-5a77-44b6-a2af-f8c7b605972f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:26.073184 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:26.073184 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:26.073184 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:26.073184 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:26.073184 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:26.073184 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:26.073184 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:26.073184 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:26.073184 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:26.073184 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:26.073184 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:26.073184 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:26.073184 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:26.073184 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:26.074035 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:26.074035 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:26.074035 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:26.074035 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:26.074035 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:26.074035 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:26.074035 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:26.074035 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:26.074035 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:26.074035 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:26.074035 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:26.074035 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:26.074035 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:26.074035 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:26.074035 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:26.074035 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:26.074035 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:26.074892 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:26.074892 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:26.074892 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:26.074892 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:26.074892 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 227/462] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:26.076037 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b6d1a-5a77-44b6-a2af-f8c7b605972f None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:26.105741 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4335c056-bb05-4679-9a0a-5dd0869f31a9 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:26.142728 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce76f2a2-4825-4f2b-9e0e-93172505cc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:26.143530 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76f2a2-4825-4f2b-9e0e-93172505cc3c None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:26.143840 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76f2a2-4825-4f2b-9e0e-93172505cc3c None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:26.144113 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76f2a2-4825-4f2b-9e0e-93172505cc3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:26.154757 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce76f2a2-4825-4f2b-9e0e-93172505cc3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:26.156253 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce76f2a2-4825-4f2b-9e0e-93172505cc3c None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:26.175168 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-ce76f2a2-4825-4f2b-9e0e-93172505cc3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:26.175168 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:26.175168 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:26.175168 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:26.175168 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:26.175168 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:26.175168 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:26.175168 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:26.175168 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:26.175168 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:26.175168 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:26.175168 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:26.175168 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:26.175168 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:26.175168 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:26.176080 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:26.176080 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:26.176080 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:26.176080 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:26.176080 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:26.176080 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:26.176080 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:26.176080 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:26.176080 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:26.176080 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:26.176080 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:26.176080 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:26.176080 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:26.176080 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:26.176080 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:26.176080 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:26.176080 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:26.176941 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:26.176941 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:26.176941 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:26.176941 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:26.176941 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 236/463] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:26.177631 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d654d3-32f4-4255-8e6b-e5f8f1b73789 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:26.177917 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d654d3-32f4-4255-8e6b-e5f8f1b73789 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:26.178194 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d654d3-32f4-4255-8e6b-e5f8f1b73789 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:26.179629 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 237/464] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:26.180160 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d654d3-32f4-4255-8e6b-e5f8f1b73789 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:26.186783 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b6d1a-5a77-44b6-a2af-f8c7b605972f None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:26.223232 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca6961-97d7-4e1c-9183-dd7ebf048cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:26.223981 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca6961-97d7-4e1c-9183-dd7ebf048cc3 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:26.224264 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca6961-97d7-4e1c-9183-dd7ebf048cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:26.224541 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca6961-97d7-4e1c-9183-dd7ebf048cc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:26.236320 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcca6961-97d7-4e1c-9183-dd7ebf048cc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:26.237767 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcca6961-97d7-4e1c-9183-dd7ebf048cc3 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:26.255814 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-dcca6961-97d7-4e1c-9183-dd7ebf048cc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:26.255814 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:26.255814 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:26.255814 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:26.255814 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:26.255814 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:26.255814 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:26.255814 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:26.255814 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:26.255814 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:26.255814 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:26.255814 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:26.255814 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:26.255814 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:26.255814 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:26.256699 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:26.256699 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:26.256699 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:26.256699 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:26.256699 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:26.256699 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:26.256699 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:26.256699 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:26.256699 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:26.256699 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:26.256699 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:26.256699 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:26.256699 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:26.256699 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:26.256699 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:26.256699 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:26.256699 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:26.257566 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:26.257566 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:26.257566 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:26.257566 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:26.257566 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 228/465] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:26.257850 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca6961-97d7-4e1c-9183-dd7ebf048cc3 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:26.291615 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d654d3-32f4-4255-8e6b-e5f8f1b73789 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:26.294616 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:26.317982 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:26.328879 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6a44c5-3b2f-43ba-af2e-b86d50054126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:26.329594 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a44c5-3b2f-43ba-af2e-b86d50054126 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:26.333045 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a44c5-3b2f-43ba-af2e-b86d50054126 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:26.333045 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a44c5-3b2f-43ba-af2e-b86d50054126 None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:26.333045 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6a44c5-3b2f-43ba-af2e-b86d50054126 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:26.333045 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6a44c5-3b2f-43ba-af2e-b86d50054126 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:26.333045 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6a44c5-3b2f-43ba-af2e-b86d50054126 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:26.350205 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 238/466] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:26.353595 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d88f7-2441-4ef4-af89-a6289007352a None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:26.353882 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d88f7-2441-4ef4-af89-a6289007352a None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:26.354157 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d88f7-2441-4ef4-af89-a6289007352a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:26.371892 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-447d88f7-2441-4ef4-af89-a6289007352a None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:26.372693 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca6961-97d7-4e1c-9183-dd7ebf048cc3 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:26.373095 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-447d88f7-2441-4ef4-af89-a6289007352a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:26.408794 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd7e67d-e313-4787-8f45-571b6acd51f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:26.410511 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd7e67d-e313-4787-8f45-571b6acd51f3 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:26.410972 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd7e67d-e313-4787-8f45-571b6acd51f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:26.411418 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd7e67d-e313-4787-8f45-571b6acd51f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:26.412447 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd7e67d-e313-4787-8f45-571b6acd51f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:26.412930 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd7e67d-e313-4787-8f45-571b6acd51f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:26.414897 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd7e67d-e313-4787-8f45-571b6acd51f3 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:26.429099 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 229/467] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:26.429922 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd7e67d-e313-4787-8f45-571b6acd51f3 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:26.446934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 239/468] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:26.457042 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447d88f7-2441-4ef4-af89-a6289007352a None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:26.537611 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd7e67d-e313-4787-8f45-571b6acd51f3 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:26.552265 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:26.558380 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:26.565715 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447d88f7-2441-4ef4-af89-a6289007352a None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:26.574181 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cde866-79b6-4a87-87b4-acbe888ae6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:26.574835 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cde866-79b6-4a87-87b4-acbe888ae6a5 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:26.575115 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cde866-79b6-4a87-87b4-acbe888ae6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:26.575390 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cde866-79b6-4a87-87b4-acbe888ae6a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:26.576170 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cde866-79b6-4a87-87b4-acbe888ae6a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:26.576477 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cde866-79b6-4a87-87b4-acbe888ae6a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:26.577852 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cde866-79b6-4a87-87b4-acbe888ae6a5 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:26.591700 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 230/469] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:26.591700 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cde866-79b6-4a87-87b4-acbe888ae6a5 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:26.607026 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ade6d-4aeb-4576-bbce-757757df34a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:26.607745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ade6d-4aeb-4576-bbce-757757df34a8 None admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:26.608027 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ade6d-4aeb-4576-bbce-757757df34a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:26.608299 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ade6d-4aeb-4576-bbce-757757df34a8 None admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:26.608998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8ade6d-4aeb-4576-bbce-757757df34a8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:26.611083 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8ade6d-4aeb-4576-bbce-757757df34a8 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:26.695133 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cde866-79b6-4a87-87b4-acbe888ae6a5 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:26.718595 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 240/470] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Jul 7 15:15:26 2022] POST /identity/v3/users => generated 246 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:15:26.728713 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ade6d-4aeb-4576-bbce-757757df34a8 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:26.728915 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba371ffc-160f-4b5d-a761-a05b8f2c73c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:26.729543 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba371ffc-160f-4b5d-a761-a05b8f2c73c0 None admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:26.729818 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba371ffc-160f-4b5d-a761-a05b8f2c73c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:26.730089 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba371ffc-160f-4b5d-a761-a05b8f2c73c0 None admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:26.730793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba371ffc-160f-4b5d-a761-a05b8f2c73c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:26.732272 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba371ffc-160f-4b5d-a761-a05b8f2c73c0 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:26.819699 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 231/471] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Jul 7 15:15:26 2022] POST /identity/v3/users => generated 248 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:15:26.819699 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba371ffc-160f-4b5d-a761-a05b8f2c73c0 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:26.857424 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:26.877422 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ade6d-4aeb-4576-bbce-757757df34a8 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:26.928470 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eebd617-f427-4f72-8d53-500b99820112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:26.929291 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebd617-f427-4f72-8d53-500b99820112 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:26.929578 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebd617-f427-4f72-8d53-500b99820112 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:26.929856 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebd617-f427-4f72-8d53-500b99820112 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:26.944846 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eebd617-f427-4f72-8d53-500b99820112 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:26.946326 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eebd617-f427-4f72-8d53-500b99820112 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:26.966343 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba371ffc-160f-4b5d-a761-a05b8f2c73c0 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:26.973026 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-0eebd617-f427-4f72-8d53-500b99820112 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:26.973026 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:26.973026 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:26.973026 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:26.973026 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:26.973026 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:26.973026 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:26.973026 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:26.973026 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:26.973026 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:26.973026 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:26.973026 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:26.973026 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:26.973026 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:26.973026 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:26.973966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:26.973966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:26.973966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:26.973966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:26.973966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:26.973966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:26.973966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:26.973966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:26.973966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:26.973966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:26.973966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:26.973966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:26.973966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:26.973966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:26.973966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:26.973966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:26.973966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:26.975190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:26.975190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:26.975190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:26.975190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:26.975190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 241/472] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:26.975798 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248e7d7-456a-4e05-9b3f-d76d65b6dd45 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:26.976020 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248e7d7-456a-4e05-9b3f-d76d65b6dd45 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:26.976299 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248e7d7-456a-4e05-9b3f-d76d65b6dd45 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:26.978332 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 242/473] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:26.983080 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2248e7d7-456a-4e05-9b3f-d76d65b6dd45 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:26.986010 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:27.012584 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e442182-b783-49e4-88a1-52040b2d17c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:27.013529 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e442182-b783-49e4-88a1-52040b2d17c3 None admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:27.013827 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e442182-b783-49e4-88a1-52040b2d17c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:27.014104 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e442182-b783-49e4-88a1-52040b2d17c3 None admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:27.014812 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e442182-b783-49e4-88a1-52040b2d17c3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:27.016854 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e442182-b783-49e4-88a1-52040b2d17c3 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:27.104334 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2248e7d7-456a-4e05-9b3f-d76d65b6dd45 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:27.136174 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 232/474] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Jul 7 15:15:26 2022] POST /identity/v3/users => generated 248 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:15:27.137877 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9f1ae-eaa1-435a-a7ab-fb7c6d122a27 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:27.138333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9f1ae-eaa1-435a-a7ab-fb7c6d122a27 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:27.138773 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9f1ae-eaa1-435a-a7ab-fb7c6d122a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:27.157520 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059733b-7a2d-435f-b3fa-8cf9d042dc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:27.158953 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059733b-7a2d-435f-b3fa-8cf9d042dc9f None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:27.159251 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059733b-7a2d-435f-b3fa-8cf9d042dc9f None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:27.159529 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059733b-7a2d-435f-b3fa-8cf9d042dc9f None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:27.160478 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1059733b-7a2d-435f-b3fa-8cf9d042dc9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:27.160794 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1059733b-7a2d-435f-b3fa-8cf9d042dc9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:27.162593 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1059733b-7a2d-435f-b3fa-8cf9d042dc9f None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:27.165676 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-56e9f1ae-eaa1-435a-a7ab-fb7c6d122a27 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:27.167313 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-56e9f1ae-eaa1-435a-a7ab-fb7c6d122a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:27.192124 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 243/475] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:27.197472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059733b-7a2d-435f-b3fa-8cf9d042dc9f None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:27.261864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 233/476] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:27.297364 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e9f1ae-eaa1-435a-a7ab-fb7c6d122a27 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:27.309277 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:27.322748 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:27.381241 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059733b-7a2d-435f-b3fa-8cf9d042dc9f None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:27.422521 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d96b101-5831-42a1-a2cf-c06475b30cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:27.423203 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96b101-5831-42a1-a2cf-c06475b30cfa None admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:27.423486 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96b101-5831-42a1-a2cf-c06475b30cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:27.423812 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96b101-5831-42a1-a2cf-c06475b30cfa None admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:27.424540 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d96b101-5831-42a1-a2cf-c06475b30cfa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:27.426059 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d96b101-5831-42a1-a2cf-c06475b30cfa None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:27.428924 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e9f1ae-eaa1-435a-a7ab-fb7c6d122a27 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:27.468838 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb1541-1d47-49eb-a64f-2fc64bd6eb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:27.469898 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb1541-1d47-49eb-a64f-2fc64bd6eb4a None admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:27.470390 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb1541-1d47-49eb-a64f-2fc64bd6eb4a None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:27.471649 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb1541-1d47-49eb-a64f-2fc64bd6eb4a None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:27.472873 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fb1541-1d47-49eb-a64f-2fc64bd6eb4a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:27.474457 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fb1541-1d47-49eb-a64f-2fc64bd6eb4a None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:27.502096 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 244/477] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Jul 7 15:15:27 2022] POST /identity/v3/users => generated 249 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:15:27.522416 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.sql.core [None req-19fb1541-1d47-49eb-a64f-2fc64bd6eb4a None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 07 15:15:27.522416 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 07 15:15:27.522416 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [parameters: {'id': 'd8dc8bd421fe45c8844937ebf9e65043', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 07 15:15:27.522416 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80300) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 07 15:15:27.541870 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-19fb1541-1d47-49eb-a64f-2fc64bd6eb4a None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 07 15:15:27.541870 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:27.541870 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 07 15:15:27.541870 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 07 15:15:27.541870 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 07 15:15:27.541870 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 07 15:15:27.541870 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 07 15:15:27.541870 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application result = self._query(query) Jul 07 15:15:27.541870 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 07 15:15:27.541870 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application conn.query(q) Jul 07 15:15:27.541870 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 07 15:15:27.541870 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 07 15:15:27.541870 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 07 15:15:27.541870 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application result.read() Jul 07 15:15:27.541870 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 07 15:15:27.542853 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 07 15:15:27.542853 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 07 15:15:27.542853 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 07 15:15:27.542853 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 07 15:15:27.542853 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 07 15:15:27.542853 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 07 15:15:27.542853 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 07 15:15:27.542853 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 07 15:15:27.542853 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:27.542853 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 07 15:15:27.542853 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:27.542853 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:27.542853 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 07 15:15:27.542853 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 07 15:15:27.542853 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 07 15:15:27.542853 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return ref.to_dict() Jul 07 15:15:27.542853 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 07 15:15:27.542853 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application next(self.gen) Jul 07 15:15:27.543936 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 07 15:15:27.543936 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application yield resource Jul 07 15:15:27.543936 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 07 15:15:27.543936 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application next(self.gen) Jul 07 15:15:27.543936 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 07 15:15:27.543936 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self.session.rollback() Jul 07 15:15:27.543936 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 07 15:15:27.543936 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self.force_reraise() Jul 07 15:15:27.543936 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 07 15:15:27.543936 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise self.value Jul 07 15:15:27.543936 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 07 15:15:27.543936 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 07 15:15:27.543936 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 07 15:15:27.543936 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application session.commit() Jul 07 15:15:27.543936 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 07 15:15:27.543936 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 07 15:15:27.543936 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 07 15:15:27.544830 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self._prepare_impl() Jul 07 15:15:27.544830 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 07 15:15:27.544830 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self.session.flush() Jul 07 15:15:27.544830 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 07 15:15:27.544830 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self._flush(objects) Jul 07 15:15:27.544830 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 07 15:15:27.544830 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 07 15:15:27.544830 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 07 15:15:27.544830 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application compat.raise_( Jul 07 15:15:27.544830 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 07 15:15:27.544830 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception Jul 07 15:15:27.544830 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 07 15:15:27.544830 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application flush_context.execute() Jul 07 15:15:27.544830 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 07 15:15:27.544830 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rec.execute(self) Jul 07 15:15:27.544830 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 07 15:15:27.544830 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 07 15:15:27.546131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 07 15:15:27.546131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 07 15:15:27.546131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 07 15:15:27.546131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 07 15:15:27.546131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 07 15:15:27.546131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 07 15:15:27.546131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 07 15:15:27.546131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 07 15:15:27.546131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 07 15:15:27.546131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 07 15:15:27.546131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 07 15:15:27.546131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 07 15:15:27.546131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 07 15:15:27.546131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 07 15:15:27.546131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 07 15:15:27.546131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception Jul 07 15:15:27.547375 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 07 15:15:27.547375 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 07 15:15:27.547375 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 07 15:15:27.547375 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 07 15:15:27.547375 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 07 15:15:27.547375 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application result = self._query(query) Jul 07 15:15:27.547375 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 07 15:15:27.547375 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application conn.query(q) Jul 07 15:15:27.547375 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 07 15:15:27.547375 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 07 15:15:27.547375 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 07 15:15:27.547375 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application result.read() Jul 07 15:15:27.547375 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 07 15:15:27.547375 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 07 15:15:27.547375 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 07 15:15:27.547375 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 07 15:15:27.548496 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 07 15:15:27.548496 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 07 15:15:27.548496 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 07 15:15:27.548496 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 07 15:15:27.548496 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 07 15:15:27.548496 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 07 15:15:27.548496 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [parameters: {'id': 'd8dc8bd421fe45c8844937ebf9e65043', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 07 15:15:27.548496 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 07 15:15:27.548496 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:27.548496 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 15:15:27.548496 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:27.548496 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:27.548496 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:27.548496 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:27.548496 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:27.548496 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:27.549655 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:27.549655 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:27.549655 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:27.549655 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:27.549655 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:27.549655 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:27.549655 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 07 15:15:27.549655 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 07 15:15:27.549655 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:27.549655 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:27.549655 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 07 15:15:27.549655 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 07 15:15:27.549655 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 07 15:15:27.549655 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 07 15:15:27.549655 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 07 15:15:27.549655 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:27.552703 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 234/478] 10.222.0.25 () {60 vars in 1131 bytes} [Thu Jul 7 15:15:27 2022] POST /identity/v3/roles => generated 144 bytes in 256 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:15:27.559631 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d96b101-5831-42a1-a2cf-c06475b30cfa None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:27.692281 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d96b101-5831-42a1-a2cf-c06475b30cfa None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:27.713103 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:27.737118 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57679f2-93cc-4300-8264-d394203198f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:27.738750 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57679f2-93cc-4300-8264-d394203198f5 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:27.739258 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57679f2-93cc-4300-8264-d394203198f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:27.739768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57679f2-93cc-4300-8264-d394203198f5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:27.755705 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57679f2-93cc-4300-8264-d394203198f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:27.765209 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-b57679f2-93cc-4300-8264-d394203198f5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 15:15:27.765209 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:27.765209 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:27.765209 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:27.765209 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:27.765209 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:27.765209 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:27.765209 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:27.765209 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:27.765209 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:27.765209 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:27.765209 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:27.765209 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:27.765209 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:27.765209 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:27.765209 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:27.766150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:27.766150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:27.766150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:27.766150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:27.766150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:27.766150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:27.766150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:27.766150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:27.766150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:27.766150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:27.766150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:27.766150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:27.766150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:27.766150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:27.766150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:27.766150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:27.766998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:27.766998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:27.766998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:27.766998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:27.766998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:27.766998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:27.766998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:27.766998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:27.766998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:27.766998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:27.766998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 15:15:27.766998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:27.769776 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 245/479] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Jul 7 15:15:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:27.776270 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb1541-1d47-49eb-a64f-2fc64bd6eb4a None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:27.862735 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:27.888327 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb1541-1d47-49eb-a64f-2fc64bd6eb4a None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:27.930156 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c78f062-55a0-4074-81cb-d6bf61844b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:27.931199 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78f062-55a0-4074-81cb-d6bf61844b59 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:27.931734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78f062-55a0-4074-81cb-d6bf61844b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:27.932208 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78f062-55a0-4074-81cb-d6bf61844b59 None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:27.933213 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c78f062-55a0-4074-81cb-d6bf61844b59 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:27.933717 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c78f062-55a0-4074-81cb-d6bf61844b59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:27.935265 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c78f062-55a0-4074-81cb-d6bf61844b59 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:27.953418 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 235/480] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:15:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:29.043303 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2e114-1dc4-46e7-b465-94d099c3bd0d None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:29.043825 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2e114-1dc4-46e7-b465-94d099c3bd0d None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:29.043899 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2e114-1dc4-46e7-b465-94d099c3bd0d None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:29.045477 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 246/481] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:29.051355 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8a136-ce69-4bb4-b93d-2bd6918a1dda None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:29.051816 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8a136-ce69-4bb4-b93d-2bd6918a1dda None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:29.051993 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8a136-ce69-4bb4-b93d-2bd6918a1dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:29.072887 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-35f8a136-ce69-4bb4-b93d-2bd6918a1dda None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:29.074183 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-35f8a136-ce69-4bb4-b93d-2bd6918a1dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:29.166203 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 236/482] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:29.207278 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e2e114-1dc4-46e7-b465-94d099c3bd0d None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:29.343012 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e2e114-1dc4-46e7-b465-94d099c3bd0d None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:29.377883 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd1d94-cc55-46ca-ab23-2de733c77703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:29.378682 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd1d94-cc55-46ca-ab23-2de733c77703 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:29.379099 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd1d94-cc55-46ca-ab23-2de733c77703 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:29.379373 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd1d94-cc55-46ca-ab23-2de733c77703 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:29.389734 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bd1d94-cc55-46ca-ab23-2de733c77703 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:29.398034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-29bd1d94-cc55-46ca-ab23-2de733c77703 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:15:29.398034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:29.398034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:29.398034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:29.398034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:29.398034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:29.398034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:29.398034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:29.398034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:29.398034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:29.398034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:29.398034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:29.398034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:29.398034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:29.398034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:29.398034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:29.398918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:29.398918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:29.398918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:29.398918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:29.398918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:29.398918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:29.398918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:29.398918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:29.398918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:29.398918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:29.398918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:29.398918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:29.398918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:29.398918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:29.398918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:29.398918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:29.399780 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:29.399780 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:29.399780 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:29.399780 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:29.399780 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:29.399780 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:29.399780 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:29.399780 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:29.399780 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:29.399780 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:29.399780 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:15:29.399780 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:29.399780 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 247/483] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Jul 7 15:15:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:29.407281 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f8a136-ce69-4bb4-b93d-2bd6918a1dda None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:29.501359 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:29.535522 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f8a136-ce69-4bb4-b93d-2bd6918a1dda None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:29.550453 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67edbf-0350-45c2-ba63-8b3caf0846cc None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:29.550739 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67edbf-0350-45c2-ba63-8b3caf0846cc None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:29.551016 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67edbf-0350-45c2-ba63-8b3caf0846cc None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:29.552547 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 248/484] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:15:29.557915 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d1950-379a-4537-94c9-fd259ecf3cc2 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:29.558639 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d1950-379a-4537-94c9-fd259ecf3cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:29.559126 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d1950-379a-4537-94c9-fd259ecf3cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:29.578521 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-d18d1950-379a-4537-94c9-fd259ecf3cc2 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:29.580416 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-d18d1950-379a-4537-94c9-fd259ecf3cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:29.598998 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6668d2c-9bd6-4e48-b481-f34a4bf9436c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:29.599621 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6668d2c-9bd6-4e48-b481-f34a4bf9436c None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:29.599910 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6668d2c-9bd6-4e48-b481-f34a4bf9436c None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:29.600185 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6668d2c-9bd6-4e48-b481-f34a4bf9436c None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:29.600964 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6668d2c-9bd6-4e48-b481-f34a4bf9436c None admin] RBAC: Adding query filter params (name=service) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:29.601273 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6668d2c-9bd6-4e48-b481-f34a4bf9436c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:29.602823 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6668d2c-9bd6-4e48-b481-f34a4bf9436c None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:29.619694 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 237/485] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Jul 7 15:15:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:29.627102 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6668d2c-9bd6-4e48-b481-f34a4bf9436c None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:29.639906 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:29.701835 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 249/486] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 07 15:15:29.754470 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18d1950-379a-4537-94c9-fd259ecf3cc2 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:29.784743 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6668d2c-9bd6-4e48-b481-f34a4bf9436c None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:29.848441 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:29.865181 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff022413-fc47-4bc5-8d69-bca73f6a2522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:29.865181 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff022413-fc47-4bc5-8d69-bca73f6a2522 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:29.865181 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff022413-fc47-4bc5-8d69-bca73f6a2522 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:29.865181 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff022413-fc47-4bc5-8d69-bca73f6a2522 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:29.875744 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18d1950-379a-4537-94c9-fd259ecf3cc2 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:29.885133 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff022413-fc47-4bc5-8d69-bca73f6a2522 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:29.886590 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff022413-fc47-4bc5-8d69-bca73f6a2522 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:29.911486 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f7851d-6f3d-4ffc-8579-53a5b7b01319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:29.912313 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f7851d-6f3d-4ffc-8579-53a5b7b01319 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:29.912594 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f7851d-6f3d-4ffc-8579-53a5b7b01319 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:29.912864 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f7851d-6f3d-4ffc-8579-53a5b7b01319 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:29.920895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-ff022413-fc47-4bc5-8d69-bca73f6a2522 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:29.920895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:29.920895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:29.920895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:29.920895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:29.920895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:29.920895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:29.920895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:29.920895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:29.920895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:29.920895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:29.920895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:29.920895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:29.920895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:29.920895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:29.921806 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:29.921806 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:29.921806 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:29.921806 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:29.921806 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:29.921806 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:29.921806 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:29.921806 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:29.921806 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:29.921806 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:29.921806 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:29.921806 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:29.921806 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:29.921806 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:29.921806 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:29.921806 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:29.921806 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:29.923255 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:29.923255 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:29.923255 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:29.923255 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:29.924097 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f7851d-6f3d-4ffc-8579-53a5b7b01319 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:29.925759 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 238/487] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:29.932439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-95f7851d-6f3d-4ffc-8579-53a5b7b01319 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:15:29.932439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:29.932439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:29.932439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:29.932439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:29.932439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:29.932439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:29.932439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:29.932439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:29.932439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:29.932439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:29.932439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:29.932439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:29.932439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:29.932439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:29.932439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:29.933344 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:29.933344 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:29.933344 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:29.933344 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:29.933344 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:29.933344 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:29.933344 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:29.933344 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:29.933344 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:29.933344 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:29.933344 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:29.933344 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:29.933344 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:29.933344 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:29.933344 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:29.933344 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:29.934157 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:29.934157 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:29.934157 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:29.934157 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:29.934157 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:29.934157 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:29.934157 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:29.934157 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:29.934157 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:29.934157 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:29.934157 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:15:29.934157 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:29.934157 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 250/488] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Jul 7 15:15:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:15:29.935410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff022413-fc47-4bc5-8d69-bca73f6a2522 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:29.939610 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f7851d-6f3d-4ffc-8579-53a5b7b01319 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:29.970413 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:30.058785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f7851d-6f3d-4ffc-8579-53a5b7b01319 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:30.086069 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff022413-fc47-4bc5-8d69-bca73f6a2522 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:30.101333 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4105154-4f1f-4d93-9e67-73fb04dede61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:30.102007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4105154-4f1f-4d93-9e67-73fb04dede61 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:30.102301 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4105154-4f1f-4d93-9e67-73fb04dede61 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:30.102882 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4105154-4f1f-4d93-9e67-73fb04dede61 None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:30.103944 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4105154-4f1f-4d93-9e67-73fb04dede61 None admin] RBAC: Adding query filter params (name=service) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:30.104263 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4105154-4f1f-4d93-9e67-73fb04dede61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:30.105647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4105154-4f1f-4d93-9e67-73fb04dede61 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:30.119725 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 251/489] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Jul 7 15:15:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:30.121373 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c7e920-ebae-4064-8d07-0314d8ba1032 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:30.121664 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c7e920-ebae-4064-8d07-0314d8ba1032 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:30.121942 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c7e920-ebae-4064-8d07-0314d8ba1032 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:30.123322 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 252/490] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:30.123920 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c7e920-ebae-4064-8d07-0314d8ba1032 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:30.137973 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dfa8d0-2dd2-4ea0-9c23-0a602c45c6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:30.138999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dfa8d0-2dd2-4ea0-9c23-0a602c45c6a3 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:30.139483 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dfa8d0-2dd2-4ea0-9c23-0a602c45c6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:30.139993 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dfa8d0-2dd2-4ea0-9c23-0a602c45c6a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:30.141471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dfa8d0-2dd2-4ea0-9c23-0a602c45c6a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:30.141977 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dfa8d0-2dd2-4ea0-9c23-0a602c45c6a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:30.143552 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dfa8d0-2dd2-4ea0-9c23-0a602c45c6a3 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:30.156932 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 239/491] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:30.158468 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4976c-450b-4d69-bde2-09c274bdcb34 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:30.158949 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4976c-450b-4d69-bde2-09c274bdcb34 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:30.159436 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4976c-450b-4d69-bde2-09c274bdcb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:30.176377 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-76b4976c-450b-4d69-bde2-09c274bdcb34 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:30.177882 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-76b4976c-450b-4d69-bde2-09c274bdcb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:30.221859 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:30.234443 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:30.245789 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c7e920-ebae-4064-8d07-0314d8ba1032 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:30.249946 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 240/492] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:30.250866 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b4976c-450b-4d69-bde2-09c274bdcb34 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:30.284514 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842abec2-c209-4cc4-848c-5e3e32225c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:30.285714 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842abec2-c209-4cc4-848c-5e3e32225c1f None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:30.286269 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842abec2-c209-4cc4-848c-5e3e32225c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:30.286547 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842abec2-c209-4cc4-848c-5e3e32225c1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:30.300218 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842abec2-c209-4cc4-848c-5e3e32225c1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:30.303109 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842abec2-c209-4cc4-848c-5e3e32225c1f None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:30.327245 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-842abec2-c209-4cc4-848c-5e3e32225c1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:30.327245 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:30.327245 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:30.327245 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:30.327245 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:30.327245 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:30.327245 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:30.327245 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:30.327245 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:30.327245 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:30.327245 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:30.327245 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:30.327245 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:30.327245 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:30.327245 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:30.328918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:30.328918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:30.328918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:30.328918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:30.328918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:30.328918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:30.328918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:30.328918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:30.328918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:30.328918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:30.328918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:30.328918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:30.328918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:30.328918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:30.328918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:30.328918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:30.328918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:30.329793 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:30.329793 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:30.329793 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:30.329793 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:30.329793 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 253/493] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:30.330634 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc761d7-8c56-40a9-b8d1-d4c8368d0a02 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:30.330921 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc761d7-8c56-40a9-b8d1-d4c8368d0a02 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:30.331197 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc761d7-8c56-40a9-b8d1-d4c8368d0a02 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:30.333092 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 254/494] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:30.334289 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc761d7-8c56-40a9-b8d1-d4c8368d0a02 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:30.362896 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b4976c-450b-4d69-bde2-09c274bdcb34 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:30.400812 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b67398-1fb0-44ad-8b58-f89ef31666e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:30.401520 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b67398-1fb0-44ad-8b58-f89ef31666e6 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:30.401800 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b67398-1fb0-44ad-8b58-f89ef31666e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:30.402069 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b67398-1fb0-44ad-8b58-f89ef31666e6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:30.409227 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b67398-1fb0-44ad-8b58-f89ef31666e6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:30.410664 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b67398-1fb0-44ad-8b58-f89ef31666e6 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:30.426347 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-52b67398-1fb0-44ad-8b58-f89ef31666e6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 07 15:15:30.426347 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:30.426347 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:30.426347 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:30.426347 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:30.426347 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:30.426347 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:30.426347 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:30.426347 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:30.426347 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:30.426347 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:30.426347 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:30.426347 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 15:15:30.426347 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 15:15:30.426347 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 15:15:30.426347 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 15:15:30.427286 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:30.427286 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:30.427286 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 15:15:30.427286 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:15:30.427286 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 15:15:30.427286 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:15:30.427286 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:30.427286 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:30.427286 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:30.427286 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:30.427286 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:30.427286 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:30.427286 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:30.427286 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:30.427286 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:30.427286 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:30.427286 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:30.427286 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:30.428263 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:30.428263 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:30.428263 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 15:15:30.428263 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 15:15:30.428263 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 15:15:30.428263 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 15:15:30.428263 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 15:15:30.428263 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 15:15:30.428263 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 07 15:15:30.428263 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:30.428263 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 241/495] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Jul 7 15:15:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:30.428868 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b67398-1fb0-44ad-8b58-f89ef31666e6 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:30.453833 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc761d7-8c56-40a9-b8d1-d4c8368d0a02 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:30.467944 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:30.491872 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f43e34-fe8c-4f17-8897-d3a35bece171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:30.492663 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f43e34-fe8c-4f17-8897-d3a35bece171 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:30.492947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f43e34-fe8c-4f17-8897-d3a35bece171 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:30.493215 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f43e34-fe8c-4f17-8897-d3a35bece171 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:30.505304 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f43e34-fe8c-4f17-8897-d3a35bece171 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:30.513584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-89f43e34-fe8c-4f17-8897-d3a35bece171 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:15:30.513584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:30.513584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:30.513584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:30.513584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:30.513584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:30.513584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:30.513584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:30.513584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:30.513584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:30.513584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:30.513584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:30.513584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:30.513584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:30.513584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:30.513584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:30.514502 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:30.514502 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:30.514502 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:30.514502 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:30.514502 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:30.514502 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:30.514502 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:30.514502 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:30.514502 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:30.514502 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:30.514502 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:30.514502 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:30.514502 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:30.514502 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:30.514502 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:30.514502 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:30.515314 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:30.515314 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:30.515314 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:30.515314 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:30.515314 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:30.515314 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:30.515314 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:30.515314 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:30.515314 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:30.515314 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:30.515314 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:15:30.515314 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:30.515314 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 255/496] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Jul 7 15:15:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:30.517338 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72114a-5c1b-41e4-b571-63e623168a2c None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:30.517638 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72114a-5c1b-41e4-b571-63e623168a2c None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:30.517946 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72114a-5c1b-41e4-b571-63e623168a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:30.523626 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:30.535717 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-9c72114a-5c1b-41e4-b571-63e623168a2c None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:30.536917 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-9c72114a-5c1b-41e4-b571-63e623168a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:30.539137 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b67398-1fb0-44ad-8b58-f89ef31666e6 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:30.573377 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca50a1-4243-4e3c-a355-12b6fa2fb958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:30.574032 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca50a1-4243-4e3c-a355-12b6fa2fb958 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:30.574317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca50a1-4243-4e3c-a355-12b6fa2fb958 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:30.574576 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca50a1-4243-4e3c-a355-12b6fa2fb958 None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:30.575296 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ca50a1-4243-4e3c-a355-12b6fa2fb958 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:30.575621 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ca50a1-4243-4e3c-a355-12b6fa2fb958 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:30.576961 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ca50a1-4243-4e3c-a355-12b6fa2fb958 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:30.588465 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 242/497] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:30.589238 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca50a1-4243-4e3c-a355-12b6fa2fb958 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:30.606681 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 256/498] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:30.607500 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c72114a-5c1b-41e4-b571-63e623168a2c None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:30.679758 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:30.686743 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca50a1-4243-4e3c-a355-12b6fa2fb958 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:30.710028 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c72114a-5c1b-41e4-b571-63e623168a2c None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:30.720990 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fe611f-378d-4a66-9e85-7cb5d4b4bf57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:30.721649 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe611f-378d-4a66-9e85-7cb5d4b4bf57 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:30.721931 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe611f-378d-4a66-9e85-7cb5d4b4bf57 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:30.722206 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe611f-378d-4a66-9e85-7cb5d4b4bf57 None admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:30.723296 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fe611f-378d-4a66-9e85-7cb5d4b4bf57 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:30.723634 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fe611f-378d-4a66-9e85-7cb5d4b4bf57 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:30.725073 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fe611f-378d-4a66-9e85-7cb5d4b4bf57 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:30.738949 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:30.743676 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b0fec5-b2f2-46c8-b97d-6187a26ba1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:30.744361 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0fec5-b2f2-46c8-b97d-6187a26ba1c1 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:30.744639 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0fec5-b2f2-46c8-b97d-6187a26ba1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:30.744915 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0fec5-b2f2-46c8-b97d-6187a26ba1c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:30.745652 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b0fec5-b2f2-46c8-b97d-6187a26ba1c1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:30.745953 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b0fec5-b2f2-46c8-b97d-6187a26ba1c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:30.747307 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b0fec5-b2f2-46c8-b97d-6187a26ba1c1 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:30.755975 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 257/499] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Jul 7 15:15:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:30.757173 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9817b5-4600-450a-9c48-477fed3801fd None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:30.757455 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9817b5-4600-450a-9c48-477fed3801fd None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:30.757723 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9817b5-4600-450a-9c48-477fed3801fd None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:30.759075 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 258/500] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:30.760332 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9817b5-4600-450a-9c48-477fed3801fd None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:30.767025 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 243/501] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Jul 7 15:15:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:30.767025 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fe611f-378d-4a66-9e85-7cb5d4b4bf57 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:30.860316 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9817b5-4600-450a-9c48-477fed3801fd None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:30.862020 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fe611f-378d-4a66-9e85-7cb5d4b4bf57 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:30.894079 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d104014-9b81-4891-a6f1-41e0ab26c535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:30.894854 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d104014-9b81-4891-a6f1-41e0ab26c535 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:30.895180 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d104014-9b81-4891-a6f1-41e0ab26c535 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:30.895417 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d104014-9b81-4891-a6f1-41e0ab26c535 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:30.896900 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5203ea-c064-41e5-9c40-eeeb819472cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:30.897692 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5203ea-c064-41e5-9c40-eeeb819472cc None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:30.897972 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5203ea-c064-41e5-9c40-eeeb819472cc None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:30.898243 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5203ea-c064-41e5-9c40-eeeb819472cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:30.908348 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d104014-9b81-4891-a6f1-41e0ab26c535 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:30.909462 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5203ea-c064-41e5-9c40-eeeb819472cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:30.909810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d104014-9b81-4891-a6f1-41e0ab26c535 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:30.917111 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-be5203ea-c064-41e5-9c40-eeeb819472cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:15:30.917111 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:30.917111 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:30.917111 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:30.917111 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:30.917111 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:30.917111 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:30.917111 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:30.917111 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:30.917111 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:30.917111 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:30.917111 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:30.917111 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:30.917111 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:30.917111 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:30.917111 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:30.918023 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:30.918023 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:30.918023 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:30.918023 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:30.918023 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:30.918023 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:30.918023 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:30.918023 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:30.918023 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:30.918023 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:30.918023 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:30.918023 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:30.918023 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:30.918023 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:30.918023 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:30.918023 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:30.918831 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:30.918831 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:30.918831 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:30.918831 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:30.918831 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:30.918831 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:30.918831 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:30.918831 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:30.918831 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:30.918831 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:30.918831 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:15:30.918831 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:30.918831 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 244/502] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Jul 7 15:15:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:30.919632 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5203ea-c064-41e5-9c40-eeeb819472cc None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:30.925847 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-4d104014-9b81-4891-a6f1-41e0ab26c535 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 07 15:15:30.925847 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:30.925847 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:30.925847 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:30.925847 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:30.925847 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:30.925847 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:30.925847 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:30.925847 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:30.925847 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:30.925847 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:30.925847 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:30.925847 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 15:15:30.925847 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 15:15:30.925847 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 15:15:30.925847 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 15:15:30.926768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:30.926768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:30.926768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 15:15:30.926768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:15:30.926768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 15:15:30.926768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:15:30.926768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:30.926768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:30.926768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:30.926768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:30.926768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:30.926768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:30.926768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:30.926768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:30.926768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:30.926768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:30.926768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:30.926768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:30.927733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:30.927733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:30.927733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 15:15:30.927733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 15:15:30.927733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 15:15:30.927733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 15:15:30.927733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 15:15:30.927733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 15:15:30.927733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 07 15:15:30.927733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:30.927733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 259/503] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:15:30 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:30.928805 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ebca72-12f9-423e-8c24-679ab0564be9 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:30.928805 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ebca72-12f9-423e-8c24-679ab0564be9 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:30.929064 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ebca72-12f9-423e-8c24-679ab0564be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:30.935851 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:30.946616 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-c1ebca72-12f9-423e-8c24-679ab0564be9 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:30.948412 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-c1ebca72-12f9-423e-8c24-679ab0564be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:30.998094 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:31.016446 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 260/504] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:31.017239 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ebca72-12f9-423e-8c24-679ab0564be9 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:31.033217 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5203ea-c064-41e5-9c40-eeeb819472cc None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:31.069073 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54028f46-321c-41af-836c-0e6c905c479f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:31.069845 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54028f46-321c-41af-836c-0e6c905c479f None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:31.070121 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54028f46-321c-41af-836c-0e6c905c479f None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:31.070392 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54028f46-321c-41af-836c-0e6c905c479f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:31.080209 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54028f46-321c-41af-836c-0e6c905c479f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:31.081638 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54028f46-321c-41af-836c-0e6c905c479f None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:31.099573 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-54028f46-321c-41af-836c-0e6c905c479f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:31.099573 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:31.099573 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:31.099573 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:31.099573 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:31.099573 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:31.099573 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:31.099573 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:31.099573 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:31.099573 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:31.099573 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:31.099573 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:31.099573 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:31.099573 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:31.099573 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:31.100902 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:31.100902 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:31.100902 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:31.100902 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:31.100902 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:31.100902 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:31.100902 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:31.100902 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:31.100902 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:31.100902 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:31.100902 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:31.100902 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:31.100902 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:31.100902 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:31.100902 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:31.100902 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:31.100902 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:31.101764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:31.101764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:31.101764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:31.101764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:31.101764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 245/505] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:31.101764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54028f46-321c-41af-836c-0e6c905c479f None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:31.117545 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ebca72-12f9-423e-8c24-679ab0564be9 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:31.149901 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ada95a-afd4-44c1-8084-22b06280c992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:31.150537 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ada95a-afd4-44c1-8084-22b06280c992 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:31.150815 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ada95a-afd4-44c1-8084-22b06280c992 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:31.151085 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ada95a-afd4-44c1-8084-22b06280c992 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:31.162058 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:31.198758 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54028f46-321c-41af-836c-0e6c905c479f None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:31.202181 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:31.217108 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ada95a-afd4-44c1-8084-22b06280c992 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:31.219117 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ada95a-afd4-44c1-8084-22b06280c992 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:31.231852 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a93f50c-1f92-4646-afba-f4b255726b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:31.232499 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93f50c-1f92-4646-afba-f4b255726b51 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:31.232779 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93f50c-1f92-4646-afba-f4b255726b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:31.233051 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93f50c-1f92-4646-afba-f4b255726b51 None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:31.233793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a93f50c-1f92-4646-afba-f4b255726b51 None admin] RBAC: Adding query filter params (name=service) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:31.234098 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a93f50c-1f92-4646-afba-f4b255726b51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:31.236034 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a93f50c-1f92-4646-afba-f4b255726b51 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:31.245505 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 246/506] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Jul 7 15:15:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:31.246268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a93f50c-1f92-4646-afba-f4b255726b51 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:31.315498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 261/507] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:15:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 15:15:31.316342 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ada95a-afd4-44c1-8084-22b06280c992 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:31.345375 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a93f50c-1f92-4646-afba-f4b255726b51 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:31.370195 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:31.378633 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2902276-ab50-40df-901d-aad1b7eb8e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:31.379291 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2902276-ab50-40df-901d-aad1b7eb8e68 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:31.379588 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2902276-ab50-40df-901d-aad1b7eb8e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:31.379881 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2902276-ab50-40df-901d-aad1b7eb8e68 None admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:31.380711 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2902276-ab50-40df-901d-aad1b7eb8e68 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:31.381021 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2902276-ab50-40df-901d-aad1b7eb8e68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:31.382443 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2902276-ab50-40df-901d-aad1b7eb8e68 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:31.397564 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:31.399409 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 247/508] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Jul 7 15:15:31 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:31.400231 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2902276-ab50-40df-901d-aad1b7eb8e68 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:31.414444 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ada95a-afd4-44c1-8084-22b06280c992 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:31.449853 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96151f7-7e1d-4560-a1da-339ee1409ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:31.450569 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96151f7-7e1d-4560-a1da-339ee1409ba0 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:31.450851 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96151f7-7e1d-4560-a1da-339ee1409ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:31.451125 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96151f7-7e1d-4560-a1da-339ee1409ba0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:31.461851 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96151f7-7e1d-4560-a1da-339ee1409ba0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:31.463292 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96151f7-7e1d-4560-a1da-339ee1409ba0 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:31.481068 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-a96151f7-7e1d-4560-a1da-339ee1409ba0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:31.481068 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:31.481068 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:31.481068 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:31.481068 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:31.481068 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:31.481068 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:31.481068 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:31.481068 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:31.481068 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:31.481068 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:31.481068 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:31.481068 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:31.481068 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:31.481068 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:31.482090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:31.482090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:31.482090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:31.482090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:31.482090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:31.482090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:31.482090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:31.482090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:31.482090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:31.482090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:31.482090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:31.482090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:31.482090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:31.482090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:31.482090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:31.482090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:31.482090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:31.483154 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:31.483154 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:31.483154 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:31.483154 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:31.483154 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 262/509] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:31.483154 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96151f7-7e1d-4560-a1da-339ee1409ba0 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:31.501121 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2902276-ab50-40df-901d-aad1b7eb8e68 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:31.534655 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1d681-101d-4cc8-89ac-6e81ae4702a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:31.535301 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1d681-101d-4cc8-89ac-6e81ae4702a9 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:31.535603 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1d681-101d-4cc8-89ac-6e81ae4702a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:31.535880 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1d681-101d-4cc8-89ac-6e81ae4702a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:31.536596 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e1d681-101d-4cc8-89ac-6e81ae4702a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:31.536901 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e1d681-101d-4cc8-89ac-6e81ae4702a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:31.538269 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e1d681-101d-4cc8-89ac-6e81ae4702a9 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:31.549307 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 248/510] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:31.550090 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1d681-101d-4cc8-89ac-6e81ae4702a9 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:31.578655 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96151f7-7e1d-4560-a1da-339ee1409ba0 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:31.598114 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:31.612459 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89658885-b995-4195-a132-91dea485ef09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:31.613255 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89658885-b995-4195-a132-91dea485ef09 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:31.614664 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89658885-b995-4195-a132-91dea485ef09 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:31.614946 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89658885-b995-4195-a132-91dea485ef09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:31.619752 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:31.629343 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89658885-b995-4195-a132-91dea485ef09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:31.630853 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89658885-b995-4195-a132-91dea485ef09 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:31.649791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-89658885-b995-4195-a132-91dea485ef09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:31.649791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:31.649791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:31.649791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:31.649791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:31.649791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:31.649791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:31.649791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:31.649791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:31.649791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:31.649791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:31.649791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:31.649791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:31.649791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:31.649791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:31.650667 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:31.650667 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:31.650667 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:31.650667 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:31.650667 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:31.650667 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:31.650667 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:31.650667 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:31.650667 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:31.650667 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:31.650667 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:31.650667 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:31.650667 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:31.650667 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:31.650667 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:31.650667 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:31.650667 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:31.656854 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:31.656854 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:31.656854 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:31.656854 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:31.657158 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1d681-101d-4cc8-89ac-6e81ae4702a9 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:31.657244 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 263/511] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:31.657244 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89658885-b995-4195-a132-91dea485ef09 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:31.689609 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6bde1-b4c6-4872-9819-75a6fc66a604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:31.690394 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6bde1-b4c6-4872-9819-75a6fc66a604 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:31.690677 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6bde1-b4c6-4872-9819-75a6fc66a604 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:31.690968 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6bde1-b4c6-4872-9819-75a6fc66a604 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:31.701319 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd6bde1-b4c6-4872-9819-75a6fc66a604 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:31.702707 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd6bde1-b4c6-4872-9819-75a6fc66a604 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:31.720493 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-2dd6bde1-b4c6-4872-9819-75a6fc66a604 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:31.720493 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:31.720493 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:31.720493 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:31.720493 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:31.720493 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:31.720493 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:31.720493 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:31.720493 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:31.720493 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:31.720493 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:31.720493 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:31.720493 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:31.720493 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:31.720493 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:31.721507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:31.721507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:31.721507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:31.721507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:31.721507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:31.721507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:31.721507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:31.721507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:31.721507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:31.721507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:31.721507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:31.721507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:31.721507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:31.721507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:31.721507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:31.721507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:31.721507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:31.722563 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:31.722563 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:31.722563 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:31.722563 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:31.722563 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 249/512] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:31.722563 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6bde1-b4c6-4872-9819-75a6fc66a604 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:31.753847 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89658885-b995-4195-a132-91dea485ef09 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:31.789167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d8e33f-f058-4279-ac6a-7be661585ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:31.789960 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8e33f-f058-4279-ac6a-7be661585ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:31.790335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8e33f-f058-4279-ac6a-7be661585ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:31.790698 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d8e33f-f058-4279-ac6a-7be661585ce0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:31.821116 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:31.826508 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6bde1-b4c6-4872-9819-75a6fc66a604 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:31.844793 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:31.856856 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d8e33f-f058-4279-ac6a-7be661585ce0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:31.858332 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d8e33f-f058-4279-ac6a-7be661585ce0 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:31.860477 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf0925-4468-4be9-958d-d65a4fbd04a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:31.861129 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf0925-4468-4be9-958d-d65a4fbd04a0 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:31.861413 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf0925-4468-4be9-958d-d65a4fbd04a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:31.861725 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf0925-4468-4be9-958d-d65a4fbd04a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:31.862446 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cf0925-4468-4be9-958d-d65a4fbd04a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:31.862753 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cf0925-4468-4be9-958d-d65a4fbd04a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:31.864141 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cf0925-4468-4be9-958d-d65a4fbd04a0 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:31.875751 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 250/513] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:31.876544 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf0925-4468-4be9-958d-d65a4fbd04a0 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:31.957519 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 264/514] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:15:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 15:15:31.958272 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d8e33f-f058-4279-ac6a-7be661585ce0 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:31.981818 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf0925-4468-4be9-958d-d65a4fbd04a0 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:32.015463 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44eb28b-b0fb-4551-95f9-a217f48c8b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:32.016784 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44eb28b-b0fb-4551-95f9-a217f48c8b51 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:32.017065 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44eb28b-b0fb-4551-95f9-a217f48c8b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:32.017338 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44eb28b-b0fb-4551-95f9-a217f48c8b51 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:32.024855 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44eb28b-b0fb-4551-95f9-a217f48c8b51 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:32.026324 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44eb28b-b0fb-4551-95f9-a217f48c8b51 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:32.040065 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-c44eb28b-b0fb-4551-95f9-a217f48c8b51 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 07 15:15:32.040065 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:32.040065 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:32.040065 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:32.040065 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:32.040065 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:32.040065 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:32.040065 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:32.040065 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:32.040065 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:32.040065 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:32.040065 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:32.040065 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 15:15:32.040065 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 15:15:32.040065 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 15:15:32.040065 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 15:15:32.041000 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:32.041000 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:32.041000 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 15:15:32.041000 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:15:32.041000 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 15:15:32.041000 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:15:32.041000 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:32.041000 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:32.041000 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:32.041000 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:32.041000 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:32.041000 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:32.041000 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:32.041000 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:32.041000 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:32.041000 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:32.041000 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:32.041000 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:32.041928 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:32.041928 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:32.041928 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 15:15:32.041928 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 15:15:32.041928 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 15:15:32.041928 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 15:15:32.041928 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 15:15:32.041928 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 15:15:32.041928 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 07 15:15:32.041928 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:32.041928 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 251/515] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:15:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:32.042500 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44eb28b-b0fb-4551-95f9-a217f48c8b51 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:32.047525 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:32.059308 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d8e33f-f058-4279-ac6a-7be661585ce0 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:32.059730 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:32.094014 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51830132-cc09-4f32-86dc-6aeb932ac1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:32.094670 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51830132-cc09-4f32-86dc-6aeb932ac1f5 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:32.094950 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51830132-cc09-4f32-86dc-6aeb932ac1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:32.095224 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51830132-cc09-4f32-86dc-6aeb932ac1f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:32.095973 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51830132-cc09-4f32-86dc-6aeb932ac1f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:32.096277 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51830132-cc09-4f32-86dc-6aeb932ac1f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:32.097896 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51830132-cc09-4f32-86dc-6aeb932ac1f5 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:32.108791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 265/516] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:32.109566 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51830132-cc09-4f32-86dc-6aeb932ac1f5 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:32.141474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44eb28b-b0fb-4551-95f9-a217f48c8b51 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:32.174339 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c25928-032f-4fbf-b342-1a22a47a419c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:32.174984 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c25928-032f-4fbf-b342-1a22a47a419c None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:32.175261 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c25928-032f-4fbf-b342-1a22a47a419c None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:32.175532 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c25928-032f-4fbf-b342-1a22a47a419c None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:32.176274 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c25928-032f-4fbf-b342-1a22a47a419c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:32.176577 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c25928-032f-4fbf-b342-1a22a47a419c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:32.177922 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c25928-032f-4fbf-b342-1a22a47a419c None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:32.189773 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 252/517] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:32.190464 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c25928-032f-4fbf-b342-1a22a47a419c None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:32.208176 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51830132-cc09-4f32-86dc-6aeb932ac1f5 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:32.241313 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df45f468-4cb6-4a86-94a4-59c0d5733231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:32.241974 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df45f468-4cb6-4a86-94a4-59c0d5733231 None admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:32.242254 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df45f468-4cb6-4a86-94a4-59c0d5733231 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:32.242530 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df45f468-4cb6-4a86-94a4-59c0d5733231 None admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:32.243222 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df45f468-4cb6-4a86-94a4-59c0d5733231 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:32.244664 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df45f468-4cb6-4a86-94a4-59c0d5733231 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:32.268929 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:32.290430 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c25928-032f-4fbf-b342-1a22a47a419c None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:32.322791 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26a3d35-6610-4f78-84a7-d1cdd15d8d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:32.323538 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a3d35-6610-4f78-84a7-d1cdd15d8d6b None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:32.323851 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a3d35-6610-4f78-84a7-d1cdd15d8d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:32.324127 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a3d35-6610-4f78-84a7-d1cdd15d8d6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:32.334250 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26a3d35-6610-4f78-84a7-d1cdd15d8d6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:32.335709 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26a3d35-6610-4f78-84a7-d1cdd15d8d6b None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:32.352769 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-b26a3d35-6610-4f78-84a7-d1cdd15d8d6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:32.352769 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:32.352769 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:32.352769 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:32.352769 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:32.352769 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:32.352769 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:32.352769 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:32.352769 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:32.352769 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:32.352769 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:32.352769 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:32.352769 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:32.352769 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:32.352769 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:32.353793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:32.353793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:32.353793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:32.353793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:32.353793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:32.353793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:32.353793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:32.353793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:32.353793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:32.353793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:32.353793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:32.353793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:32.353793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:32.353793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:32.353793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:32.353793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:32.353793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:32.354731 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:32.354731 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:32.354731 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:32.354731 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:32.354731 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 253/518] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:32.355374 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26a3d35-6610-4f78-84a7-d1cdd15d8d6b None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:32.364323 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 266/519] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Jul 7 15:15:32 2022] POST /identity/v3/users => generated 247 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:15:32.365135 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df45f468-4cb6-4a86-94a4-59c0d5733231 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:32.368959 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:32.469915 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26a3d35-6610-4f78-84a7-d1cdd15d8d6b None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:32.479387 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df45f468-4cb6-4a86-94a4-59c0d5733231 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:32.499246 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:32.508364 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2837060-8a88-4cd5-bd7e-2f1982dd6770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:32.509031 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2837060-8a88-4cd5-bd7e-2f1982dd6770 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:32.511264 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2837060-8a88-4cd5-bd7e-2f1982dd6770 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:32.511264 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2837060-8a88-4cd5-bd7e-2f1982dd6770 None admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:32.511264 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2837060-8a88-4cd5-bd7e-2f1982dd6770 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:32.511264 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2837060-8a88-4cd5-bd7e-2f1982dd6770 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:32.512145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2837060-8a88-4cd5-bd7e-2f1982dd6770 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:32.521003 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902d906-f080-45de-8c1c-c5235abcdf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:32.521796 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902d906-f080-45de-8c1c-c5235abcdf9f None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:32.522073 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902d906-f080-45de-8c1c-c5235abcdf9f None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:32.522345 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902d906-f080-45de-8c1c-c5235abcdf9f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:32.531523 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 254/520] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Jul 7 15:15:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:32.531767 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b902d906-f080-45de-8c1c-c5235abcdf9f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:32.533248 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b902d906-f080-45de-8c1c-c5235abcdf9f None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:32.536592 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2837060-8a88-4cd5-bd7e-2f1982dd6770 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:32.550331 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-b902d906-f080-45de-8c1c-c5235abcdf9f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 07 15:15:32.550331 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:32.550331 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:32.550331 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:32.550331 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:32.550331 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:32.550331 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:32.550331 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:32.550331 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:32.550331 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:32.550331 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:32.550331 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:32.550331 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 15:15:32.550331 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 15:15:32.550331 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 15:15:32.550331 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 15:15:32.551281 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:32.551281 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:32.551281 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 15:15:32.551281 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:15:32.551281 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 15:15:32.551281 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:15:32.551281 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:32.551281 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:32.551281 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:32.551281 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:32.551281 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:32.551281 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:32.551281 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:32.551281 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:32.551281 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:32.551281 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:32.551281 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:32.551281 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:32.552253 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:32.552253 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:32.552253 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 15:15:32.552253 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 15:15:32.552253 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 15:15:32.552253 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 15:15:32.552253 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 15:15:32.552253 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 15:15:32.552253 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 07 15:15:32.552253 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:32.552253 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 267/521] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Jul 7 15:15:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:32.552944 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902d906-f080-45de-8c1c-c5235abcdf9f None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:32.619967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:32.652012 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2837060-8a88-4cd5-bd7e-2f1982dd6770 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:32.658399 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902d906-f080-45de-8c1c-c5235abcdf9f None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:32.688589 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd206825-4d5d-4950-acbd-c91be29070e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:32.689369 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd206825-4d5d-4950-acbd-c91be29070e0 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:32.689650 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd206825-4d5d-4950-acbd-c91be29070e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:32.689925 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd206825-4d5d-4950-acbd-c91be29070e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:32.696689 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd206825-4d5d-4950-acbd-c91be29070e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:32.696843 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef9f9e4-3b7f-420b-9f4c-ab5f45d6a610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:32.697425 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9f9e4-3b7f-420b-9f4c-ab5f45d6a610 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:32.697713 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9f9e4-3b7f-420b-9f4c-ab5f45d6a610 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:32.697990 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9f9e4-3b7f-420b-9f4c-ab5f45d6a610 None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:32.698157 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd206825-4d5d-4950-acbd-c91be29070e0 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:32.698897 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef9f9e4-3b7f-420b-9f4c-ab5f45d6a610 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:32.699202 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef9f9e4-3b7f-420b-9f4c-ab5f45d6a610 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:32.701032 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef9f9e4-3b7f-420b-9f4c-ab5f45d6a610 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:32.712333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-dd206825-4d5d-4950-acbd-c91be29070e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:15:32.712333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:32.712333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:32.712333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:32.712333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:32.712333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:32.712333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:32.712333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:32.712333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:32.712333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:32.712333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:32.712333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:32.712333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 15:15:32.712333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 15:15:32.712333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 15:15:32.713207 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 15:15:32.713207 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:32.713207 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:32.713207 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:32.713207 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:32.713207 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:32.713207 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:32.713207 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:32.713207 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:32.713207 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:32.713207 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:32.713207 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:32.713207 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:32.713207 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:32.713207 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:32.713207 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:32.713207 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:32.714056 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 15:15:32.714056 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 15:15:32.714056 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 15:15:32.714056 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 15:15:32.714056 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 15:15:32.714056 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 15:15:32.714056 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:15:32.714056 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:32.714567 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 268/522] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:32.714567 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef9f9e4-3b7f-420b-9f4c-ab5f45d6a610 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:32.720834 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 255/523] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:15:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:32.720834 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd206825-4d5d-4950-acbd-c91be29070e0 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:32.773877 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:32.823431 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef9f9e4-3b7f-420b-9f4c-ab5f45d6a610 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:32.829692 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd206825-4d5d-4950-acbd-c91be29070e0 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:32.850964 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:32.862732 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b104cc-2d16-4027-909d-36af5e89dd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:32.863387 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b104cc-2d16-4027-909d-36af5e89dd52 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:32.863714 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b104cc-2d16-4027-909d-36af5e89dd52 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:32.863988 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b104cc-2d16-4027-909d-36af5e89dd52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:32.865111 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0710e1-2dae-4478-98d0-9c531df05e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:32.865781 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0710e1-2dae-4478-98d0-9c531df05e34 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:32.866068 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0710e1-2dae-4478-98d0-9c531df05e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:32.866344 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0710e1-2dae-4478-98d0-9c531df05e34 None admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:32.867193 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0710e1-2dae-4478-98d0-9c531df05e34 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:32.868148 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0710e1-2dae-4478-98d0-9c531df05e34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:32.868950 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0710e1-2dae-4478-98d0-9c531df05e34 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:32.887497 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 256/524] 10.222.0.25 () {56 vars in 1139 bytes} [Thu Jul 7 15:15:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:32.889106 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0710e1-2dae-4478-98d0-9c531df05e34 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:32.936899 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b104cc-2d16-4027-909d-36af5e89dd52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:32.938909 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b104cc-2d16-4027-909d-36af5e89dd52 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:33.006776 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0710e1-2dae-4478-98d0-9c531df05e34 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:33.009077 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:33.042294 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf6d966-af11-4896-8728-831eec2e61da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:33.043087 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf6d966-af11-4896-8728-831eec2e61da None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:33.043371 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf6d966-af11-4896-8728-831eec2e61da None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:33.043676 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf6d966-af11-4896-8728-831eec2e61da None admin] PATH_INFO: `/v3/projects/service` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:33.051239 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf6d966-af11-4896-8728-831eec2e61da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:33.056841 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf6d966-af11-4896-8728-831eec2e61da None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:33.056972 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 269/525] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:15:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 15:15:33.056972 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b104cc-2d16-4027-909d-36af5e89dd52 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:33.071595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-5cf6d966-af11-4896-8728-831eec2e61da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:15:33.071595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:33.071595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:33.071595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:33.071595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:33.071595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:33.071595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:33.071595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:33.071595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:33.071595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:33.071595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:33.071595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:33.071595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 15:15:33.071595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 15:15:33.071595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 15:15:33.072566 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 15:15:33.072566 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:33.072566 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:33.072566 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:33.072566 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:33.072566 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:33.072566 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:33.072566 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:33.072566 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:33.072566 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:33.072566 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:33.072566 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:33.072566 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:33.072566 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:33.072566 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:33.072566 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:33.072566 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:33.073406 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 15:15:33.073406 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 15:15:33.073406 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 15:15:33.073406 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 15:15:33.073406 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 15:15:33.073406 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 15:15:33.073406 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:15:33.073406 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:33.073406 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 257/526] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:15:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:33.073869 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf6d966-af11-4896-8728-831eec2e61da None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:33.090080 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:33.168590 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b104cc-2d16-4027-909d-36af5e89dd52 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:33.185410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf6d966-af11-4896-8728-831eec2e61da None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:33.206080 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d18f6-6f37-49e3-8a6a-3a0f96fd7e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:33.206735 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d18f6-6f37-49e3-8a6a-3a0f96fd7e02 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:33.207021 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d18f6-6f37-49e3-8a6a-3a0f96fd7e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:33.207298 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d18f6-6f37-49e3-8a6a-3a0f96fd7e02 None admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:33.208189 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23d18f6-6f37-49e3-8a6a-3a0f96fd7e02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:33.208673 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23d18f6-6f37-49e3-8a6a-3a0f96fd7e02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:33.209914 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23d18f6-6f37-49e3-8a6a-3a0f96fd7e02 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:33.220055 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b3ed1-5c5b-4300-b432-e82e081d1146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:33.220702 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b3ed1-5c5b-4300-b432-e82e081d1146 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:33.220979 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b3ed1-5c5b-4300-b432-e82e081d1146 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:33.221253 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b3ed1-5c5b-4300-b432-e82e081d1146 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:33.233452 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 270/527] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Jul 7 15:15:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:33.233452 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d18f6-6f37-49e3-8a6a-3a0f96fd7e02 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:33.259329 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:33.297994 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0b3ed1-5c5b-4300-b432-e82e081d1146 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:33.299439 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0b3ed1-5c5b-4300-b432-e82e081d1146 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:33.319617 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:33.335455 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d18f6-6f37-49e3-8a6a-3a0f96fd7e02 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:33.371529 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe84946-d47a-4a0b-9724-632a72a7689e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:33.372346 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe84946-d47a-4a0b-9724-632a72a7689e None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:33.372742 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe84946-d47a-4a0b-9724-632a72a7689e None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:33.373019 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe84946-d47a-4a0b-9724-632a72a7689e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:33.384152 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe84946-d47a-4a0b-9724-632a72a7689e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:33.385985 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe84946-d47a-4a0b-9724-632a72a7689e None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:33.404176 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-5fe84946-d47a-4a0b-9724-632a72a7689e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:33.404176 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:33.404176 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:33.404176 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:33.404176 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:33.404176 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:33.404176 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:33.404176 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:33.404176 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:33.404176 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:33.404176 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:33.404176 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:33.404176 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:33.404176 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:33.404176 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:33.405086 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:33.405086 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:33.405086 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:33.405086 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:33.405086 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:33.405086 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:33.405086 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:33.405086 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:33.405086 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:33.405086 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:33.405086 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:33.405086 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:33.405086 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:33.405086 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:33.405086 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:33.405086 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:33.405086 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:33.405972 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:33.405972 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:33.405972 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:33.405972 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:33.405972 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 271/528] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:33.406263 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe84946-d47a-4a0b-9724-632a72a7689e None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:33.407178 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 258/529] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:15:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 15:15:33.408239 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b3ed1-5c5b-4300-b432-e82e081d1146 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:33.477761 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:33.519684 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe84946-d47a-4a0b-9724-632a72a7689e None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:33.519860 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b3ed1-5c5b-4300-b432-e82e081d1146 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:33.553516 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3711b48-2a25-4be1-b839-099c11751b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:33.554049 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26906df4-46f3-4781-a561-2e813f82d2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:33.554193 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3711b48-2a25-4be1-b839-099c11751b79 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:33.554437 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3711b48-2a25-4be1-b839-099c11751b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:33.554713 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3711b48-2a25-4be1-b839-099c11751b79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:33.555431 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26906df4-46f3-4781-a561-2e813f82d2a4 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:33.555542 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3711b48-2a25-4be1-b839-099c11751b79 None admin] RBAC: Adding query filter params (role.id=3d4f4e0a472e4ff9ac68d90c8612e956, scope.project.id=c5260c7fd3614e9e93b167c86a94260e, user.id=de1632555cb345f2b9213da1925c02f4) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:33.555875 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3711b48-2a25-4be1-b839-099c11751b79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:33.556247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26906df4-46f3-4781-a561-2e813f82d2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:33.556705 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26906df4-46f3-4781-a561-2e813f82d2a4 None admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:33.557308 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3711b48-2a25-4be1-b839-099c11751b79 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:33.557879 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26906df4-46f3-4781-a561-2e813f82d2a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:33.558397 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26906df4-46f3-4781-a561-2e813f82d2a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:33.560049 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26906df4-46f3-4781-a561-2e813f82d2a4 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:33.563131 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:33.563611 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 259/530] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:15:33 2022] GET /identity/v3/role_assignments?user.id=de1632555cb345f2b9213da1925c02f4&scope.project.id=c5260c7fd3614e9e93b167c86a94260e&role.id=3d4f4e0a472e4ff9ac68d90c8612e956 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:33.564605 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3711b48-2a25-4be1-b839-099c11751b79 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:33.586700 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 272/531] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Jul 7 15:15:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:33.587581 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26906df4-46f3-4781-a561-2e813f82d2a4 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:33.698727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3711b48-2a25-4be1-b839-099c11751b79 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:33.724589 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26906df4-46f3-4781-a561-2e813f82d2a4 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:33.737037 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:33.749491 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fedcb3-3fec-46f7-9d6e-04869415291d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:33.755349 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fedcb3-3fec-46f7-9d6e-04869415291d None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:33.755349 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fedcb3-3fec-46f7-9d6e-04869415291d None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:33.755349 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fedcb3-3fec-46f7-9d6e-04869415291d None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:33.755349 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fedcb3-3fec-46f7-9d6e-04869415291d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:33.755349 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fedcb3-3fec-46f7-9d6e-04869415291d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:33.755831 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fedcb3-3fec-46f7-9d6e-04869415291d None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:33.768792 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 260/532] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:33.770316 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fedcb3-3fec-46f7-9d6e-04869415291d None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:33.776113 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3146c0-b966-4ce9-81eb-8219e51918bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:33.777103 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3146c0-b966-4ce9-81eb-8219e51918bc None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:33.777525 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3146c0-b966-4ce9-81eb-8219e51918bc None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:33.777935 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3146c0-b966-4ce9-81eb-8219e51918bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:33.789759 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3146c0-b966-4ce9-81eb-8219e51918bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:33.791584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3146c0-b966-4ce9-81eb-8219e51918bc None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:33.810219 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-2f3146c0-b966-4ce9-81eb-8219e51918bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:33.810219 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:33.810219 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:33.810219 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:33.810219 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:33.810219 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:33.810219 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:33.810219 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:33.810219 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:33.810219 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:33.810219 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:33.810219 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:33.810219 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:33.810219 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:33.810219 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:33.811247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:33.811247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:33.811247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:33.811247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:33.811247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:33.811247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:33.811247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:33.811247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:33.811247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:33.811247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:33.811247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:33.811247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:33.811247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:33.811247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:33.811247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:33.811247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:33.811247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:33.812860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:33.812860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:33.812860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:33.812860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:33.814882 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 273/533] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:33.815934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3146c0-b966-4ce9-81eb-8219e51918bc None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:33.870901 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:33.883385 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fedcb3-3fec-46f7-9d6e-04869415291d None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:33.929211 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf72da-2ce7-4626-8812-5f8398c9e680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:33.929993 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf72da-2ce7-4626-8812-5f8398c9e680 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:33.930340 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf72da-2ce7-4626-8812-5f8398c9e680 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:33.930972 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf72da-2ce7-4626-8812-5f8398c9e680 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:33.931107 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3146c0-b966-4ce9-81eb-8219e51918bc None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:33.931691 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babf72da-2ce7-4626-8812-5f8398c9e680 None admin] RBAC: Adding query filter params (role.id=3d4f4e0a472e4ff9ac68d90c8612e956, scope.project.id=c5260c7fd3614e9e93b167c86a94260e, user.id=3d0d29e718f446e2a9fd1b2575a86041) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:33.932071 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babf72da-2ce7-4626-8812-5f8398c9e680 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:33.934563 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babf72da-2ce7-4626-8812-5f8398c9e680 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:33.941972 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 261/534] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:15:33 2022] GET /identity/v3/role_assignments?user.id=3d0d29e718f446e2a9fd1b2575a86041&scope.project.id=c5260c7fd3614e9e93b167c86a94260e&role.id=3d4f4e0a472e4ff9ac68d90c8612e956 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:33.943187 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf72da-2ce7-4626-8812-5f8398c9e680 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:33.995610 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f157e-b3dc-40f9-8446-c44f2d8f14fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:33.996427 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f157e-b3dc-40f9-8446-c44f2d8f14fb None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:33.996706 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f157e-b3dc-40f9-8446-c44f2d8f14fb None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:33.996979 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f157e-b3dc-40f9-8446-c44f2d8f14fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:34.016305 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520f157e-b3dc-40f9-8446-c44f2d8f14fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:34.017814 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520f157e-b3dc-40f9-8446-c44f2d8f14fb None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:34.047673 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-520f157e-b3dc-40f9-8446-c44f2d8f14fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:15:34.047673 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:34.047673 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:34.047673 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:34.047673 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:34.047673 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:34.047673 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:34.047673 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:34.047673 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:34.047673 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:34.047673 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:34.047673 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:34.047673 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 15:15:34.047673 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 15:15:34.047673 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 15:15:34.048574 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 15:15:34.048574 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:34.048574 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:34.048574 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:34.048574 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:34.048574 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:34.048574 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:34.048574 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:34.048574 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:34.048574 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:34.048574 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:34.048574 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:34.048574 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:34.048574 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:34.048574 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:34.048574 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:34.048574 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:34.049408 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 15:15:34.049408 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 15:15:34.049408 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 15:15:34.049408 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 15:15:34.049408 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 15:15:34.049408 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 15:15:34.049408 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:15:34.049408 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:34.049989 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:34.052158 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 274/535] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:15:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:34.057782 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f157e-b3dc-40f9-8446-c44f2d8f14fb None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:34.105246 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf72da-2ce7-4626-8812-5f8398c9e680 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:34.151718 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e5bcf-85bf-455b-9bf7-4d4d701af3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:34.152393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e5bcf-85bf-455b-9bf7-4d4d701af3d9 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:34.152672 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e5bcf-85bf-455b-9bf7-4d4d701af3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:34.152942 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e5bcf-85bf-455b-9bf7-4d4d701af3d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:34.153681 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2e5bcf-85bf-455b-9bf7-4d4d701af3d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:34.153992 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2e5bcf-85bf-455b-9bf7-4d4d701af3d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:34.155356 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2e5bcf-85bf-455b-9bf7-4d4d701af3d9 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:34.170243 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 262/536] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:34.175991 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e5bcf-85bf-455b-9bf7-4d4d701af3d9 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:34.239164 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f157e-b3dc-40f9-8446-c44f2d8f14fb None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:34.263183 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:34.317742 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20f13d0-008f-457b-a796-85b5c18fb21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:34.318325 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f13d0-008f-457b-a796-85b5c18fb21c None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:34.318605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f13d0-008f-457b-a796-85b5c18fb21c None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:34.318875 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f13d0-008f-457b-a796-85b5c18fb21c None admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:34.319693 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20f13d0-008f-457b-a796-85b5c18fb21c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:34.320002 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20f13d0-008f-457b-a796-85b5c18fb21c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:34.321440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20f13d0-008f-457b-a796-85b5c18fb21c None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:34.349204 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 275/537] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Jul 7 15:15:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:15:34.352253 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e5bcf-85bf-455b-9bf7-4d4d701af3d9 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:34.357339 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20f13d0-008f-457b-a796-85b5c18fb21c None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:34.393363 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:34.450233 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4546d449-0167-488f-9c42-281387170c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:34.451340 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546d449-0167-488f-9c42-281387170c05 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:34.454818 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546d449-0167-488f-9c42-281387170c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:34.455275 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546d449-0167-488f-9c42-281387170c05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:34.465149 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4546d449-0167-488f-9c42-281387170c05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:34.467488 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4546d449-0167-488f-9c42-281387170c05 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:34.482016 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20f13d0-008f-457b-a796-85b5c18fb21c None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:34.492088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-4546d449-0167-488f-9c42-281387170c05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:15:34.492088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:34.492088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:34.492088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:34.492088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:34.492088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:34.492088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:34.492088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:34.492088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:34.492088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:34.492088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:34.492088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:34.492088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 15:15:34.492088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 15:15:34.492088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 15:15:34.492914 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 15:15:34.492914 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:34.492914 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:34.492914 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:34.492914 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:34.492914 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:34.492914 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:34.492914 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:34.492914 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:34.492914 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:34.492914 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:34.492914 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:34.492914 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:34.492914 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:34.492914 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:34.492914 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:34.492914 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:34.493762 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 15:15:34.493762 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 15:15:34.493762 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 15:15:34.493762 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 15:15:34.493762 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 15:15:34.493762 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 15:15:34.493762 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:15:34.493762 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:34.496274 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 263/538] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:15:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:34.504425 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4546d449-0167-488f-9c42-281387170c05 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:34.545439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454480e0-f16c-4c11-8038-0f6cd18cb2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:34.546183 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454480e0-f16c-4c11-8038-0f6cd18cb2b1 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:34.547088 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454480e0-f16c-4c11-8038-0f6cd18cb2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:34.547439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454480e0-f16c-4c11-8038-0f6cd18cb2b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:34.548366 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454480e0-f16c-4c11-8038-0f6cd18cb2b1 None admin] RBAC: Adding query filter params (role.id=3d4f4e0a472e4ff9ac68d90c8612e956, scope.project.id=c5260c7fd3614e9e93b167c86a94260e, user.id=2a0fbebb3dd2458ba4fe5d7f0ce211b8) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:34.548745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454480e0-f16c-4c11-8038-0f6cd18cb2b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:34.550645 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454480e0-f16c-4c11-8038-0f6cd18cb2b1 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:34.556688 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 276/539] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:15:34 2022] GET /identity/v3/role_assignments?user.id=2a0fbebb3dd2458ba4fe5d7f0ce211b8&scope.project.id=c5260c7fd3614e9e93b167c86a94260e&role.id=3d4f4e0a472e4ff9ac68d90c8612e956 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:34.558158 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59223b-3b81-451f-ba6c-8b262b75ec1d None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:34.558492 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59223b-3b81-451f-ba6c-8b262b75ec1d None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:34.558773 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59223b-3b81-451f-ba6c-8b262b75ec1d None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:34.560305 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 277/540] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:34.566628 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a50357-9381-46ca-9332-bb7c9ace9f2a None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:34.566968 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a50357-9381-46ca-9332-bb7c9ace9f2a None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:34.567188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a50357-9381-46ca-9332-bb7c9ace9f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:34.585901 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-f9a50357-9381-46ca-9332-bb7c9ace9f2a None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:34.587343 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-f9a50357-9381-46ca-9332-bb7c9ace9f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:34.618910 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:34.624060 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4546d449-0167-488f-9c42-281387170c05 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:34.661692 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df58896-cc18-412a-856b-7ffd35fbeb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:34.662740 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df58896-cc18-412a-856b-7ffd35fbeb5a None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:34.663221 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df58896-cc18-412a-856b-7ffd35fbeb5a None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:34.663716 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df58896-cc18-412a-856b-7ffd35fbeb5a None admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:34.664765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df58896-cc18-412a-856b-7ffd35fbeb5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:34.665276 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df58896-cc18-412a-856b-7ffd35fbeb5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:34.666929 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df58896-cc18-412a-856b-7ffd35fbeb5a None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:34.669234 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 278/541] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 15:15:34.689317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 264/542] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Jul 7 15:15:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:34.695631 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a50357-9381-46ca-9332-bb7c9ace9f2a None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:34.712005 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df58896-cc18-412a-856b-7ffd35fbeb5a None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:34.766377 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:34.850723 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df58896-cc18-412a-856b-7ffd35fbeb5a None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:34.869675 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a50357-9381-46ca-9332-bb7c9ace9f2a None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:34.892525 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b8a8c-0f0b-4e88-a989-7d8f9b5e5888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:34.893332 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b8a8c-0f0b-4e88-a989-7d8f9b5e5888 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:34.893672 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b8a8c-0f0b-4e88-a989-7d8f9b5e5888 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:34.893951 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b8a8c-0f0b-4e88-a989-7d8f9b5e5888 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:34.918767 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b8a8c-0f0b-4e88-a989-7d8f9b5e5888 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:34.936999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-d16b8a8c-0f0b-4e88-a989-7d8f9b5e5888 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:15:34.936999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:34.936999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:34.936999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:34.936999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:34.936999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:34.936999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:34.936999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:34.936999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:34.936999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:34.936999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:34.936999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:34.936999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:34.936999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:34.936999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:34.936999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:34.938150 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:34.938150 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:34.938150 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:34.938150 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:34.938150 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:34.938150 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:34.938150 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:34.938150 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:34.938150 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:34.938150 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:34.938150 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:34.938150 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:34.938150 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:34.938150 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:34.938150 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:34.938150 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:34.939604 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:34.939604 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:34.939604 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:34.939604 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:34.939604 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:34.939604 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:34.939604 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:34.939604 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:34.939604 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:34.939604 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:34.939604 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:15:34.939604 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:34.939604 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 265/543] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Jul 7 15:15:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:15:34.943115 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f28f95-7b22-46b1-8205-3215b6f2f184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:34.943115 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f28f95-7b22-46b1-8205-3215b6f2f184 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:34.943115 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f28f95-7b22-46b1-8205-3215b6f2f184 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:34.943115 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f28f95-7b22-46b1-8205-3215b6f2f184 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:34.947606 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f28f95-7b22-46b1-8205-3215b6f2f184 None admin] RBAC: Adding query filter params (role.id=3d4f4e0a472e4ff9ac68d90c8612e956, scope.project.id=c5260c7fd3614e9e93b167c86a94260e, user.id=e79efaea155c46ecb9831956e68aec8e) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:34.947606 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f28f95-7b22-46b1-8205-3215b6f2f184 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:34.947606 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f28f95-7b22-46b1-8205-3215b6f2f184 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:34.949035 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b8a8c-0f0b-4e88-a989-7d8f9b5e5888 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:34.954990 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 279/544] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:15:34 2022] GET /identity/v3/role_assignments?user.id=e79efaea155c46ecb9831956e68aec8e&scope.project.id=c5260c7fd3614e9e93b167c86a94260e&role.id=3d4f4e0a472e4ff9ac68d90c8612e956 => generated 259 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:35.112487 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:35.122370 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b8a8c-0f0b-4e88-a989-7d8f9b5e5888 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:35.171393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422eb20-132c-4721-b684-c3b786782b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:35.172734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422eb20-132c-4721-b684-c3b786782b03 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:35.173010 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422eb20-132c-4721-b684-c3b786782b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:35.173279 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422eb20-132c-4721-b684-c3b786782b03 None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:35.174039 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422eb20-132c-4721-b684-c3b786782b03 None admin] RBAC: Adding query filter params (name=service) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:35.174340 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422eb20-132c-4721-b684-c3b786782b03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:35.175805 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422eb20-132c-4721-b684-c3b786782b03 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:35.190236 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 266/545] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Jul 7 15:15:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:35.195585 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f28f95-7b22-46b1-8205-3215b6f2f184 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:35.240681 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:35.345115 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f28f95-7b22-46b1-8205-3215b6f2f184 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:35.388972 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ce363-25f8-4d1b-95a2-7d4d67431e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:35.389696 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ce363-25f8-4d1b-95a2-7d4d67431e9d None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:35.389976 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ce363-25f8-4d1b-95a2-7d4d67431e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:35.390795 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ce363-25f8-4d1b-95a2-7d4d67431e9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:35.407856 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8ce363-25f8-4d1b-95a2-7d4d67431e9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:35.409316 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8ce363-25f8-4d1b-95a2-7d4d67431e9d None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:35.433761 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-2e8ce363-25f8-4d1b-95a2-7d4d67431e9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:35.433761 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:35.433761 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:35.433761 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:35.433761 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:35.433761 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:35.433761 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:35.433761 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:35.433761 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:35.433761 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:35.433761 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:35.433761 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:35.433761 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:35.433761 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:35.433761 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:35.434658 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:35.434658 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:35.434658 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:35.434658 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:35.434658 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:35.434658 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:35.434658 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:35.434658 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:35.434658 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:35.434658 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:35.434658 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:35.434658 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:35.434658 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:35.434658 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:35.434658 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:35.434658 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:35.434658 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:35.436078 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:35.436078 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:35.436078 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:35.436078 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:35.436078 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 280/546] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:15:35.440827 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422eb20-132c-4721-b684-c3b786782b03 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:35.547096 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422eb20-132c-4721-b684-c3b786782b03 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:35.582452 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c05fa-7b8d-41f6-b5e6-def9125e6aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:35.583115 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c05fa-7b8d-41f6-b5e6-def9125e6aac None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:35.583397 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c05fa-7b8d-41f6-b5e6-def9125e6aac None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:35.583722 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c05fa-7b8d-41f6-b5e6-def9125e6aac None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:35.584448 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1c05fa-7b8d-41f6-b5e6-def9125e6aac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:35.584753 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1c05fa-7b8d-41f6-b5e6-def9125e6aac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:35.586120 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1c05fa-7b8d-41f6-b5e6-def9125e6aac None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:35.599444 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 267/547] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:35.604758 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ce363-25f8-4d1b-95a2-7d4d67431e9d None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:35.688075 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:35.709553 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ce363-25f8-4d1b-95a2-7d4d67431e9d None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:35.744383 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43536f49-417d-42a9-8f46-f6ffe22d86fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:35.745176 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43536f49-417d-42a9-8f46-f6ffe22d86fb None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:35.745507 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43536f49-417d-42a9-8f46-f6ffe22d86fb None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:35.745790 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43536f49-417d-42a9-8f46-f6ffe22d86fb None admin] PATH_INFO: `/v3/users/swift` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:35.753828 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43536f49-417d-42a9-8f46-f6ffe22d86fb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:35.755270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43536f49-417d-42a9-8f46-f6ffe22d86fb None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:35.770803 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-43536f49-417d-42a9-8f46-f6ffe22d86fb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 07 15:15:35.770803 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:35.770803 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:35.770803 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:35.770803 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:35.770803 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:35.770803 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:35.770803 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:35.770803 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:35.770803 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:35.770803 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:35.770803 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:35.770803 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 15:15:35.770803 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 15:15:35.770803 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 15:15:35.770803 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 15:15:35.773305 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:35.773305 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:35.773305 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 15:15:35.773305 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:15:35.773305 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 15:15:35.773305 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:15:35.773305 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:35.773305 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:35.773305 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:35.773305 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:35.773305 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:35.773305 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:35.773305 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:35.773305 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:35.773305 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:35.773305 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:35.773305 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:35.773305 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:35.775605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:35.775605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:35.775605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 15:15:35.775605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 15:15:35.775605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 15:15:35.775605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 15:15:35.775605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 15:15:35.775605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 15:15:35.775605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 07 15:15:35.775605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:35.775605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 281/548] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Jul 7 15:15:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:35.779972 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c05fa-7b8d-41f6-b5e6-def9125e6aac None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:35.813758 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:35.850618 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd22be-ded8-47d4-9320-e70f6aa70e95 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:35.850962 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd22be-ded8-47d4-9320-e70f6aa70e95 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:35.851170 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd22be-ded8-47d4-9320-e70f6aa70e95 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:35.852781 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 282/549] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:35.859013 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291739a-f7e1-4b63-be2f-4c02fcc45ae6 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:35.859301 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291739a-f7e1-4b63-be2f-4c02fcc45ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:35.859593 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291739a-f7e1-4b63-be2f-4c02fcc45ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:35.894166 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-d291739a-f7e1-4b63-be2f-4c02fcc45ae6 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:35.895380 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-d291739a-f7e1-4b63-be2f-4c02fcc45ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:35.947380 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c05fa-7b8d-41f6-b5e6-def9125e6aac None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:36.020992 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90e85f-9234-482e-8480-641643850858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:36.021694 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90e85f-9234-482e-8480-641643850858 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:36.021992 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90e85f-9234-482e-8480-641643850858 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:36.022271 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90e85f-9234-482e-8480-641643850858 None admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:36.023119 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90e85f-9234-482e-8480-641643850858 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:36.023429 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90e85f-9234-482e-8480-641643850858 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:36.024883 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90e85f-9234-482e-8480-641643850858 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:36.036985 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 283/550] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:36.050511 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 268/551] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Jul 7 15:15:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:15:36.055691 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d291739a-f7e1-4b63-be2f-4c02fcc45ae6 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:36.091629 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90e85f-9234-482e-8480-641643850858 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:36.155395 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:36.216682 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d291739a-f7e1-4b63-be2f-4c02fcc45ae6 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:36.230385 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:36.263815 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90e85f-9234-482e-8480-641643850858 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:36.303461 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c2a6d-a264-4706-be37-39fb412626af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:36.304170 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c2a6d-a264-4706-be37-39fb412626af None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:36.304535 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c2a6d-a264-4706-be37-39fb412626af None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:36.304709 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c2a6d-a264-4706-be37-39fb412626af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:36.306819 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96806227-527e-43f3-95e5-31f23623e70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:36.307653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96806227-527e-43f3-95e5-31f23623e70d None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:36.307936 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96806227-527e-43f3-95e5-31f23623e70d None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:36.308205 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96806227-527e-43f3-95e5-31f23623e70d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:36.340486 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96806227-527e-43f3-95e5-31f23623e70d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:36.348953 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-96806227-527e-43f3-95e5-31f23623e70d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:15:36.348953 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:36.348953 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:36.348953 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:36.348953 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:36.348953 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:36.348953 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:36.348953 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:36.348953 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:36.348953 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:36.348953 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:36.348953 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:36.348953 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:36.348953 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:36.348953 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:36.348953 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:36.349895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:36.349895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:36.349895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:36.349895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:36.349895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:36.349895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:36.349895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:36.349895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:36.349895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:36.349895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:36.349895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:36.349895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:36.349895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:36.349895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:36.349895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:36.349895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:36.350734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:36.350734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:36.350734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:36.350734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:36.350734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:36.350734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:36.350734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:36.350734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:36.350734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:36.350734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:36.350734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:15:36.350734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:36.350734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 269/552] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Jul 7 15:15:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:36.360756 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96806227-527e-43f3-95e5-31f23623e70d None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:36.418097 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575c2a6d-a264-4706-be37-39fb412626af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:36.420944 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575c2a6d-a264-4706-be37-39fb412626af None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:36.513496 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:36.523676 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96806227-527e-43f3-95e5-31f23623e70d None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:36.557551 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 284/553] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:15:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 15:15:36.562880 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:36.563730 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90d6874-3798-4e6a-a263-8cd5c2c42467 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:36.564226 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90d6874-3798-4e6a-a263-8cd5c2c42467 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:36.564704 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90d6874-3798-4e6a-a263-8cd5c2c42467 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:36.566571 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 285/554] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:36.567452 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90d6874-3798-4e6a-a263-8cd5c2c42467 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:36.573704 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6d0e47-e661-4bde-b395-f7398318f777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:36.574376 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d0e47-e661-4bde-b395-f7398318f777 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:36.574659 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d0e47-e661-4bde-b395-f7398318f777 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:36.575709 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d0e47-e661-4bde-b395-f7398318f777 None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:36.575806 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6d0e47-e661-4bde-b395-f7398318f777 None admin] RBAC: Adding query filter params (name=service) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:36.576118 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6d0e47-e661-4bde-b395-f7398318f777 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:36.577589 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6d0e47-e661-4bde-b395-f7398318f777 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:36.595075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 270/555] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Jul 7 15:15:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:36.595075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42034bac-0349-4afb-968f-958b11ec00e5 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:36.595075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42034bac-0349-4afb-968f-958b11ec00e5 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:36.595075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42034bac-0349-4afb-968f-958b11ec00e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:36.616699 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-42034bac-0349-4afb-968f-958b11ec00e5 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:36.617891 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-42034bac-0349-4afb-968f-958b11ec00e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:36.690650 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90d6874-3798-4e6a-a263-8cd5c2c42467 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:36.695716 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 271/556] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:36.696651 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42034bac-0349-4afb-968f-958b11ec00e5 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:36.727882 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052b516d-2718-48f0-8757-309bdedb5a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:36.729212 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b516d-2718-48f0-8757-309bdedb5a39 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:36.729713 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b516d-2718-48f0-8757-309bdedb5a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:36.730183 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b516d-2718-48f0-8757-309bdedb5a39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:36.741406 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052b516d-2718-48f0-8757-309bdedb5a39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:36.743144 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052b516d-2718-48f0-8757-309bdedb5a39 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:36.761329 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-052b516d-2718-48f0-8757-309bdedb5a39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:36.761329 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:36.761329 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:36.761329 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:36.761329 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:36.761329 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:36.761329 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:36.761329 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:36.761329 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:36.761329 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:36.761329 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:36.761329 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:36.761329 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:36.761329 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:36.761329 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:36.762333 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:36.762333 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:36.762333 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:36.762333 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:36.762333 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:36.762333 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:36.762333 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:36.762333 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:36.762333 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:36.762333 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:36.762333 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:36.762333 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:36.762333 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:36.762333 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:36.762333 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:36.762333 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:36.762333 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:36.763286 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:36.763286 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:36.763286 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:36.763286 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:36.766171 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 286/557] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:36.767234 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052b516d-2718-48f0-8757-309bdedb5a39 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:36.790890 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:36.824146 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42034bac-0349-4afb-968f-958b11ec00e5 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:36.850135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:36.862028 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ab15b-9f04-4b92-8f3b-7f0b5c1295ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:36.863388 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ab15b-9f04-4b92-8f3b-7f0b5c1295ca None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:36.863388 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ab15b-9f04-4b92-8f3b-7f0b5c1295ca None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:36.863388 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ab15b-9f04-4b92-8f3b-7f0b5c1295ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:36.874478 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4ab15b-9f04-4b92-8f3b-7f0b5c1295ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:36.876003 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4ab15b-9f04-4b92-8f3b-7f0b5c1295ca None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:36.882574 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052b516d-2718-48f0-8757-309bdedb5a39 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:36.896209 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-2e4ab15b-9f04-4b92-8f3b-7f0b5c1295ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:36.896209 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:36.896209 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:36.896209 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:36.896209 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:36.896209 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:36.896209 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:36.896209 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:36.896209 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:36.896209 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:36.896209 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:36.896209 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:36.896209 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:36.896209 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:36.896209 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:36.897906 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:36.897906 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:36.897906 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:36.897906 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:36.897906 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:36.897906 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:36.897906 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:36.897906 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:36.897906 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:36.897906 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:36.897906 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:36.897906 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:36.897906 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:36.897906 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:36.897906 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:36.897906 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:36.897906 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:36.898792 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:36.898792 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:36.898792 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:36.898792 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:36.901912 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 272/558] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:36.902779 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ab15b-9f04-4b92-8f3b-7f0b5c1295ca None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:36.925433 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4360d38-d7ca-4e23-be8c-1560de9609a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:36.926708 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4360d38-d7ca-4e23-be8c-1560de9609a1 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:36.927184 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4360d38-d7ca-4e23-be8c-1560de9609a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:36.927781 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4360d38-d7ca-4e23-be8c-1560de9609a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:36.940611 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4360d38-d7ca-4e23-be8c-1560de9609a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:36.952077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-c4360d38-d7ca-4e23-be8c-1560de9609a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:15:36.952077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:36.952077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:36.952077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:36.952077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:36.952077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:36.952077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:36.952077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:36.952077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:36.952077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:36.952077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:36.952077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:36.952077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:36.952077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:36.952077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:36.952077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:36.954967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:36.954967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:36.954967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:36.954967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:36.954967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:36.954967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:36.954967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:36.954967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:36.954967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:36.954967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:36.954967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:36.954967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:36.954967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:36.954967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:36.954967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:36.954967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:36.956270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:36.956270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:36.956270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:36.956270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:36.956270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:36.956270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:36.956270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:36.956270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:36.956270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:36.956270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:36.956270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:15:36.956270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:36.959074 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 287/559] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Jul 7 15:15:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:36.960203 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4360d38-d7ca-4e23-be8c-1560de9609a1 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:37.026317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ab15b-9f04-4b92-8f3b-7f0b5c1295ca None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:37.055947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:37.063308 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bce106-8e14-4bf4-a374-fde545483c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:37.064000 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bce106-8e14-4bf4-a374-fde545483c2d None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:37.064288 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bce106-8e14-4bf4-a374-fde545483c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:37.064567 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bce106-8e14-4bf4-a374-fde545483c2d None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:37.065374 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bce106-8e14-4bf4-a374-fde545483c2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:37.072145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bce106-8e14-4bf4-a374-fde545483c2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:37.072281 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4360d38-d7ca-4e23-be8c-1560de9609a1 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:37.072380 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bce106-8e14-4bf4-a374-fde545483c2d None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:37.080356 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 273/560] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:37.081237 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bce106-8e14-4bf4-a374-fde545483c2d None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:37.104305 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3b77b0-4157-4081-a32a-1b64654aa972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:37.105104 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b77b0-4157-4081-a32a-1b64654aa972 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:37.105483 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b77b0-4157-4081-a32a-1b64654aa972 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:37.105912 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b77b0-4157-4081-a32a-1b64654aa972 None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:37.106755 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3b77b0-4157-4081-a32a-1b64654aa972 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:37.107155 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3b77b0-4157-4081-a32a-1b64654aa972 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:37.108723 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3b77b0-4157-4081-a32a-1b64654aa972 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:37.119149 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:37.121942 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 288/561] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:37.122857 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3b77b0-4157-4081-a32a-1b64654aa972 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:37.192522 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bce106-8e14-4bf4-a374-fde545483c2d None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:37.230397 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8941131-d43e-4deb-8d27-9e2ff513c6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:37.230605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3b77b0-4157-4081-a32a-1b64654aa972 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:37.231070 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8941131-d43e-4deb-8d27-9e2ff513c6cf None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:37.231356 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8941131-d43e-4deb-8d27-9e2ff513c6cf None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:37.231663 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8941131-d43e-4deb-8d27-9e2ff513c6cf None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:37.232411 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8941131-d43e-4deb-8d27-9e2ff513c6cf None admin] RBAC: Adding query filter params (name=service) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:37.232723 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8941131-d43e-4deb-8d27-9e2ff513c6cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:37.234127 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8941131-d43e-4deb-8d27-9e2ff513c6cf None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:37.246698 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 274/562] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Jul 7 15:15:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:37.246698 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8941131-d43e-4deb-8d27-9e2ff513c6cf None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:37.268162 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e526123c-0975-47ce-894a-14eb17ceb186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:37.269477 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e526123c-0975-47ce-894a-14eb17ceb186 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:37.269761 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e526123c-0975-47ce-894a-14eb17ceb186 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:37.270036 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e526123c-0975-47ce-894a-14eb17ceb186 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:37.277452 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e526123c-0975-47ce-894a-14eb17ceb186 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:37.278840 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e526123c-0975-47ce-894a-14eb17ceb186 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:37.281705 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:37.293472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-e526123c-0975-47ce-894a-14eb17ceb186 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:15:37.293472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:37.293472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:37.293472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:37.293472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:37.293472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:37.293472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:37.293472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:37.293472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:37.293472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:37.293472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:37.293472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:37.293472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 15:15:37.293472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 15:15:37.293472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 15:15:37.294480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 15:15:37.294480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:37.294480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:37.294480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:37.294480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:37.294480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:37.294480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:37.294480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:37.294480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:37.294480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:37.294480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:37.294480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:37.294480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:37.294480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:37.294480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:37.294480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:37.294480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:37.295407 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 15:15:37.295407 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 15:15:37.295407 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 15:15:37.295407 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 15:15:37.295407 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 15:15:37.295407 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 15:15:37.295407 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:15:37.295407 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:37.295407 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 289/563] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:15:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:37.295943 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c8f10-31bc-40cd-8563-f7c8a7f2d7fd None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:37.296174 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c8f10-31bc-40cd-8563-f7c8a7f2d7fd None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:37.296570 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c8f10-31bc-40cd-8563-f7c8a7f2d7fd None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:37.297934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 290/564] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:37.298575 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c8f10-31bc-40cd-8563-f7c8a7f2d7fd None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:37.353050 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:37.356844 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8941131-d43e-4deb-8d27-9e2ff513c6cf None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:37.391699 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c07758-4435-4321-bd34-331f50957653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:37.392482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c07758-4435-4321-bd34-331f50957653 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:37.392766 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c07758-4435-4321-bd34-331f50957653 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:37.393042 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c07758-4435-4321-bd34-331f50957653 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:37.400263 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c07758-4435-4321-bd34-331f50957653 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:37.400543 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c8f10-31bc-40cd-8563-f7c8a7f2d7fd None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:37.401708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c07758-4435-4321-bd34-331f50957653 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:37.415451 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-a9c07758-4435-4321-bd34-331f50957653 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 07 15:15:37.415451 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:37.415451 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:37.415451 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:37.415451 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:37.415451 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:37.415451 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:37.415451 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:37.415451 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:37.415451 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:37.415451 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:37.415451 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:37.415451 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 15:15:37.415451 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 15:15:37.415451 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 15:15:37.415451 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 15:15:37.416626 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:37.416626 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:37.416626 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 15:15:37.416626 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:15:37.416626 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 15:15:37.416626 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:15:37.416626 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:37.416626 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:37.416626 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:37.416626 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:37.416626 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:37.416626 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:37.416626 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:37.416626 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:37.416626 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:37.416626 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:37.416626 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:37.416626 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:37.417777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:37.417777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:37.417777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 15:15:37.417777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 15:15:37.417777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 15:15:37.417777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 15:15:37.417777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 15:15:37.417777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 15:15:37.417777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 07 15:15:37.417777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:37.417777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 275/565] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Jul 7 15:15:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:37.418491 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24365d-2115-400f-a8a1-9d4183c345c9 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:37.418491 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24365d-2115-400f-a8a1-9d4183c345c9 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:37.418491 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24365d-2115-400f-a8a1-9d4183c345c9 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:37.419799 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 276/566] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:37.420387 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de24365d-2115-400f-a8a1-9d4183c345c9 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:37.440362 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ab422b-3d45-4eaf-90c4-ce307e828be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:37.441147 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ab422b-3d45-4eaf-90c4-ce307e828be0 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:37.441424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ab422b-3d45-4eaf-90c4-ce307e828be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:37.441694 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ab422b-3d45-4eaf-90c4-ce307e828be0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:37.451891 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ab422b-3d45-4eaf-90c4-ce307e828be0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:37.453952 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ab422b-3d45-4eaf-90c4-ce307e828be0 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:37.470354 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-77ab422b-3d45-4eaf-90c4-ce307e828be0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:37.470354 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:37.470354 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:37.470354 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:37.470354 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:37.470354 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:37.470354 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:37.470354 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:37.470354 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:37.470354 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:37.470354 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:37.470354 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:37.470354 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:37.470354 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:37.470354 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:37.471399 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:37.471399 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:37.471399 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:37.471399 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:37.471399 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:37.471399 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:37.471399 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:37.471399 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:37.471399 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:37.471399 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:37.471399 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:37.471399 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:37.471399 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:37.471399 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:37.471399 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:37.471399 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:37.471399 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:37.472434 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:37.472434 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:37.472434 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:37.472434 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:37.472434 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 291/567] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:37.476872 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d7346-a38d-41fa-a5f2-c56b52d508c5 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:37.477172 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d7346-a38d-41fa-a5f2-c56b52d508c5 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:37.477451 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d7346-a38d-41fa-a5f2-c56b52d508c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:37.493587 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-008d7346-a38d-41fa-a5f2-c56b52d508c5 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:37.494764 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-008d7346-a38d-41fa-a5f2-c56b52d508c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:37.524974 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de24365d-2115-400f-a8a1-9d4183c345c9 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:37.540800 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:37.559130 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b24a0e-f87b-4c91-95b1-4c0cba2a68b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:37.560009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b24a0e-f87b-4c91-95b1-4c0cba2a68b7 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:37.560118 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 292/568] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:37.560217 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b24a0e-f87b-4c91-95b1-4c0cba2a68b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:37.560384 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b24a0e-f87b-4c91-95b1-4c0cba2a68b7 None admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:37.560717 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d7346-a38d-41fa-a5f2-c56b52d508c5 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:37.561158 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b24a0e-f87b-4c91-95b1-4c0cba2a68b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:37.561464 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b24a0e-f87b-4c91-95b1-4c0cba2a68b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:37.563437 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b24a0e-f87b-4c91-95b1-4c0cba2a68b7 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:37.580349 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 277/569] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Jul 7 15:15:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:37.580349 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653de6b-6540-4ac4-b42e-d7b93e023abd None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:37.580349 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653de6b-6540-4ac4-b42e-d7b93e023abd None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:37.580349 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653de6b-6540-4ac4-b42e-d7b93e023abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:37.594017 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-1653de6b-6540-4ac4-b42e-d7b93e023abd None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:37.595234 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-1653de6b-6540-4ac4-b42e-d7b93e023abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:37.608582 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:37.664092 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 278/570] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:37.665033 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1653de6b-6540-4ac4-b42e-d7b93e023abd None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:37.674394 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d7346-a38d-41fa-a5f2-c56b52d508c5 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:37.710249 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42449d84-ddfe-49a0-96d1-8c87fb5c253a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:37.710913 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42449d84-ddfe-49a0-96d1-8c87fb5c253a None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:37.711193 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42449d84-ddfe-49a0-96d1-8c87fb5c253a None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:37.711465 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42449d84-ddfe-49a0-96d1-8c87fb5c253a None admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:37.712322 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42449d84-ddfe-49a0-96d1-8c87fb5c253a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:37.712625 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42449d84-ddfe-49a0-96d1-8c87fb5c253a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:37.714037 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42449d84-ddfe-49a0-96d1-8c87fb5c253a None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:37.750755 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 293/571] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Jul 7 15:15:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:37.751553 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42449d84-ddfe-49a0-96d1-8c87fb5c253a None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:37.763525 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1653de6b-6540-4ac4-b42e-d7b93e023abd None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:37.792297 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:37.801889 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912bb74-93af-4ea8-bbd5-bc666c52887e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:37.802624 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912bb74-93af-4ea8-bbd5-bc666c52887e None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:37.802979 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912bb74-93af-4ea8-bbd5-bc666c52887e None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:37.803325 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912bb74-93af-4ea8-bbd5-bc666c52887e None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:37.804091 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c912bb74-93af-4ea8-bbd5-bc666c52887e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:37.804401 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c912bb74-93af-4ea8-bbd5-bc666c52887e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:37.805758 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c912bb74-93af-4ea8-bbd5-bc666c52887e None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:37.817393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 279/572] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:37.818171 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912bb74-93af-4ea8-bbd5-bc666c52887e None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:37.828862 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:37.852086 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42449d84-ddfe-49a0-96d1-8c87fb5c253a None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:37.884991 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e684c-d652-4884-a397-f6dea52bda73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:37.885641 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e684c-d652-4884-a397-f6dea52bda73 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:37.885921 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e684c-d652-4884-a397-f6dea52bda73 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:37.886246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e684c-d652-4884-a397-f6dea52bda73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:37.887082 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7e684c-d652-4884-a397-f6dea52bda73 None admin] RBAC: Adding query filter params (role.id=3d4f4e0a472e4ff9ac68d90c8612e956, scope.project.id=c5260c7fd3614e9e93b167c86a94260e, user.id=b18033f849b240999d12c1885d2f7b91) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:37.887385 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7e684c-d652-4884-a397-f6dea52bda73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:37.888862 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7e684c-d652-4884-a397-f6dea52bda73 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:37.894147 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 294/573] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:15:37 2022] GET /identity/v3/role_assignments?user.id=b18033f849b240999d12c1885d2f7b91&scope.project.id=c5260c7fd3614e9e93b167c86a94260e&role.id=3d4f4e0a472e4ff9ac68d90c8612e956 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:37.894895 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e684c-d652-4884-a397-f6dea52bda73 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:37.987292 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912bb74-93af-4ea8-bbd5-bc666c52887e None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:38.027490 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498364d-3317-4411-8d7a-a0b4702d436c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:38.028433 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498364d-3317-4411-8d7a-a0b4702d436c None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:38.028711 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498364d-3317-4411-8d7a-a0b4702d436c None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:38.029094 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498364d-3317-4411-8d7a-a0b4702d436c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:38.041681 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6498364d-3317-4411-8d7a-a0b4702d436c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:38.050009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-6498364d-3317-4411-8d7a-a0b4702d436c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:15:38.050009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:38.050009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:38.050009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:38.050009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:38.050009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:38.050009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:38.050009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:38.050009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:38.050009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:38.050009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:38.050009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:38.050009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:38.050009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:38.050009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:38.050009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:38.051081 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:38.051081 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:38.051081 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:38.051081 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:38.051081 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:38.051081 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:38.051081 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:38.051081 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:38.051081 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:38.051081 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:38.051081 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:38.051081 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:38.051081 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:38.051081 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:38.051081 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:38.051081 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:38.051973 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:38.051973 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:38.051973 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:38.051973 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:38.051973 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:38.051973 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:38.051973 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:38.051973 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:38.051973 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:38.051973 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:38.051973 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:15:38.051973 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:38.051973 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 280/574] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Jul 7 15:15:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:38.051973 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498364d-3317-4411-8d7a-a0b4702d436c None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:38.073197 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e684c-d652-4884-a397-f6dea52bda73 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:38.075536 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:38.108125 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dbb564-06ca-43b8-b034-48d1c25dda57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:38.109646 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dbb564-06ca-43b8-b034-48d1c25dda57 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:38.110121 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dbb564-06ca-43b8-b034-48d1c25dda57 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:38.110572 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dbb564-06ca-43b8-b034-48d1c25dda57 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:38.121286 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dbb564-06ca-43b8-b034-48d1c25dda57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:38.121881 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:38.129199 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-f9dbb564-06ca-43b8-b034-48d1c25dda57 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:15:38.129199 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:38.129199 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:38.129199 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:38.129199 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:38.129199 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:38.129199 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:38.129199 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:38.129199 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:38.129199 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:38.129199 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:38.129199 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:38.129199 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:38.129199 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:38.129199 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:38.129199 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:38.130363 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:38.130363 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:38.130363 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:38.130363 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:38.130363 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:38.130363 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:38.130363 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:38.130363 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:38.130363 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:38.130363 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:38.130363 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:38.130363 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:38.130363 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:38.130363 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:38.130363 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:38.130363 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:38.131436 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:38.131436 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:38.131436 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:38.131436 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:38.131436 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:38.131436 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:38.131436 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:38.131436 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:38.131436 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:38.131436 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:38.131436 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:15:38.131436 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:38.135399 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 295/575] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Jul 7 15:15:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:38.136234 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dbb564-06ca-43b8-b034-48d1c25dda57 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:38.161168 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498364d-3317-4411-8d7a-a0b4702d436c None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:38.195920 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354fe54c-b691-49be-8ce6-3c07c532ab85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:38.196711 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fe54c-b691-49be-8ce6-3c07c532ab85 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:38.196989 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fe54c-b691-49be-8ce6-3c07c532ab85 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:38.197263 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fe54c-b691-49be-8ce6-3c07c532ab85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:38.208616 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354fe54c-b691-49be-8ce6-3c07c532ab85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:38.210045 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354fe54c-b691-49be-8ce6-3c07c532ab85 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:38.228072 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-354fe54c-b691-49be-8ce6-3c07c532ab85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:38.228072 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:38.228072 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:38.228072 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:38.228072 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:38.228072 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:38.228072 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:38.228072 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:38.228072 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:38.228072 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:38.228072 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:38.228072 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:38.228072 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:38.228072 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:38.228072 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:38.228992 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:38.228992 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:38.228992 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:38.228992 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:38.228992 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:38.228992 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:38.228992 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:38.228992 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:38.228992 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:38.228992 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:38.228992 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:38.228992 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:38.228992 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:38.228992 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:38.228992 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:38.228992 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:38.228992 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:38.234679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:38.234679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:38.234679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:38.234679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:38.234679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 281/576] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:38.234679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354fe54c-b691-49be-8ce6-3c07c532ab85 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:38.248350 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dbb564-06ca-43b8-b034-48d1c25dda57 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:38.284873 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a8ca1-ef5f-4afc-9a6b-f56e86b8b8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:38.285760 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a8ca1-ef5f-4afc-9a6b-f56e86b8b8ee None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:38.286046 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a8ca1-ef5f-4afc-9a6b-f56e86b8b8ee None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:38.286323 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a8ca1-ef5f-4afc-9a6b-f56e86b8b8ee None admin] PATH_INFO: `/v3/users/glance` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:38.299694 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a8ca1-ef5f-4afc-9a6b-f56e86b8b8ee None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:38.301229 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a8ca1-ef5f-4afc-9a6b-f56e86b8b8ee None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:38.315526 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-2a1a8ca1-ef5f-4afc-9a6b-f56e86b8b8ee None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 07 15:15:38.315526 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:38.315526 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:38.315526 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:38.315526 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:38.315526 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:38.315526 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:38.315526 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:38.315526 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:38.315526 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:38.315526 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:38.315526 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:38.315526 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 15:15:38.315526 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 15:15:38.315526 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 15:15:38.315526 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 15:15:38.316730 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:38.316730 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:38.316730 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 15:15:38.316730 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:15:38.316730 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 15:15:38.316730 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:15:38.316730 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:38.316730 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:38.316730 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:38.316730 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:38.316730 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:38.316730 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:38.316730 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:38.316730 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:38.316730 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:38.316730 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:38.316730 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:38.316730 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:38.317928 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:38.317928 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:38.317928 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 15:15:38.317928 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 15:15:38.317928 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 15:15:38.317928 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 15:15:38.317928 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 15:15:38.317928 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 15:15:38.317928 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 07 15:15:38.317928 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:38.319985 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 296/577] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:15:38 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:38.321000 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a8ca1-ef5f-4afc-9a6b-f56e86b8b8ee None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:38.349951 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:38.353811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354fe54c-b691-49be-8ce6-3c07c532ab85 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:38.380236 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:38.397937 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b452ca1d-5dbc-4ea6-baa9-17ba513d6f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:38.398737 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452ca1d-5dbc-4ea6-baa9-17ba513d6f49 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:38.399122 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452ca1d-5dbc-4ea6-baa9-17ba513d6f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:38.399496 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452ca1d-5dbc-4ea6-baa9-17ba513d6f49 None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:38.400401 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b452ca1d-5dbc-4ea6-baa9-17ba513d6f49 None admin] RBAC: Adding query filter params (name=service) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:38.400813 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b452ca1d-5dbc-4ea6-baa9-17ba513d6f49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:38.402716 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b452ca1d-5dbc-4ea6-baa9-17ba513d6f49 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:38.413417 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 282/578] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Jul 7 15:15:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:38.414394 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b452ca1d-5dbc-4ea6-baa9-17ba513d6f49 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:38.430981 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a8ca1-ef5f-4afc-9a6b-f56e86b8b8ee None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:38.474576 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7fedb7-d1d9-41a7-a0ba-8707ead95495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:38.475239 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7fedb7-d1d9-41a7-a0ba-8707ead95495 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:38.475520 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7fedb7-d1d9-41a7-a0ba-8707ead95495 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:38.475821 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7fedb7-d1d9-41a7-a0ba-8707ead95495 None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:38.476573 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7fedb7-d1d9-41a7-a0ba-8707ead95495 None admin] RBAC: Adding query filter params (name=service) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:38.476919 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7fedb7-d1d9-41a7-a0ba-8707ead95495 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:38.478406 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7fedb7-d1d9-41a7-a0ba-8707ead95495 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:38.490639 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 297/579] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Jul 7 15:15:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:38.490639 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7fedb7-d1d9-41a7-a0ba-8707ead95495 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:38.525631 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b452ca1d-5dbc-4ea6-baa9-17ba513d6f49 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:38.564806 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e5fb58-66a6-46e9-bc0d-d3fbea53f35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:38.565394 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5fb58-66a6-46e9-bc0d-d3fbea53f35b None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:38.565676 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5fb58-66a6-46e9-bc0d-d3fbea53f35b None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:38.565958 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5fb58-66a6-46e9-bc0d-d3fbea53f35b None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:38.566692 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e5fb58-66a6-46e9-bc0d-d3fbea53f35b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:38.567003 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e5fb58-66a6-46e9-bc0d-d3fbea53f35b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:38.568427 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e5fb58-66a6-46e9-bc0d-d3fbea53f35b None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:38.585270 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 283/580] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:38.585270 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e5fb58-66a6-46e9-bc0d-d3fbea53f35b None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:38.587082 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:38.603258 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7fedb7-d1d9-41a7-a0ba-8707ead95495 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:38.630590 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:38.639541 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c26fd-7a80-4cc6-b2d1-a5e7e997da5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:38.640866 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c26fd-7a80-4cc6-b2d1-a5e7e997da5a None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:38.641211 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c26fd-7a80-4cc6-b2d1-a5e7e997da5a None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:38.641500 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c26fd-7a80-4cc6-b2d1-a5e7e997da5a None admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:38.642347 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6c26fd-7a80-4cc6-b2d1-a5e7e997da5a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:38.642656 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6c26fd-7a80-4cc6-b2d1-a5e7e997da5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:38.644120 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6c26fd-7a80-4cc6-b2d1-a5e7e997da5a None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:38.666578 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 298/581] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Jul 7 15:15:38 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:38.666578 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c26fd-7a80-4cc6-b2d1-a5e7e997da5a None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:38.695635 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e5fb58-66a6-46e9-bc0d-d3fbea53f35b None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:38.730222 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d113a4e5-9202-4c7e-adc9-bf50540bafc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:38.731088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113a4e5-9202-4c7e-adc9-bf50540bafc9 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:38.731442 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113a4e5-9202-4c7e-adc9-bf50540bafc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:38.731813 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113a4e5-9202-4c7e-adc9-bf50540bafc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:38.742385 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d113a4e5-9202-4c7e-adc9-bf50540bafc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:38.743926 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d113a4e5-9202-4c7e-adc9-bf50540bafc9 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:38.762599 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-d113a4e5-9202-4c7e-adc9-bf50540bafc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:38.762599 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:38.762599 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:38.762599 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:38.762599 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:38.762599 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:38.762599 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:38.762599 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:38.762599 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:38.762599 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:38.762599 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:38.762599 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:38.762599 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:38.762599 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:38.762599 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:38.763780 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:38.763780 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:38.763780 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:38.763780 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:38.763780 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:38.763780 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:38.763780 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:38.763780 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:38.763780 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:38.763780 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:38.763780 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:38.763780 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:38.763780 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:38.763780 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:38.763780 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:38.763780 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:38.763780 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:38.764800 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:38.764800 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:38.764800 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:38.764800 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:38.764800 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 284/582] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:38.765960 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d113a4e5-9202-4c7e-adc9-bf50540bafc9 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:38.774876 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c26fd-7a80-4cc6-b2d1-a5e7e997da5a None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:38.808887 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb50740-56db-4d2e-8f13-dfd91a147577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:38.809677 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb50740-56db-4d2e-8f13-dfd91a147577 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:38.809960 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb50740-56db-4d2e-8f13-dfd91a147577 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:38.810235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb50740-56db-4d2e-8f13-dfd91a147577 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:38.820883 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb50740-56db-4d2e-8f13-dfd91a147577 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:38.822266 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb50740-56db-4d2e-8f13-dfd91a147577 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:38.824882 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:38.841175 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-1eb50740-56db-4d2e-8f13-dfd91a147577 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:38.841175 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:38.841175 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:38.841175 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:38.841175 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:38.841175 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:38.841175 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:38.841175 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:38.841175 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:38.841175 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:38.841175 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:38.841175 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:38.841175 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:38.841175 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:38.841175 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:38.842311 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:38.842311 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:38.842311 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:38.842311 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:38.842311 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:38.842311 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:38.842311 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:38.842311 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:38.842311 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:38.842311 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:38.842311 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:38.842311 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:38.842311 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:38.842311 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:38.842311 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:38.842311 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:38.842311 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:38.843430 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:38.843430 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:38.843430 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:38.843430 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:38.843929 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 299/583] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:38.844306 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb50740-56db-4d2e-8f13-dfd91a147577 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:38.868351 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:38.878624 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d113a4e5-9202-4c7e-adc9-bf50540bafc9 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:38.917201 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a150100-90c1-49a4-944c-d51c4e7b15e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:38.918129 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a150100-90c1-49a4-944c-d51c4e7b15e5 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:38.918487 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a150100-90c1-49a4-944c-d51c4e7b15e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:38.918844 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a150100-90c1-49a4-944c-d51c4e7b15e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:38.926498 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a150100-90c1-49a4-944c-d51c4e7b15e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:38.928119 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a150100-90c1-49a4-944c-d51c4e7b15e5 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:38.942811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-0a150100-90c1-49a4-944c-d51c4e7b15e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:15:38.942811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:38.942811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:38.942811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:38.942811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:38.942811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:38.942811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:38.942811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:38.942811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:38.942811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:38.942811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:38.942811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:38.942811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 15:15:38.942811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 15:15:38.942811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 15:15:38.943727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 15:15:38.943727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:38.943727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:38.943727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:38.943727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:38.943727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:38.943727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:38.943727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:38.943727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:38.943727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:38.943727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:38.943727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:38.943727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:38.943727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:38.943727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:38.943727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:38.943727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:38.944643 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 15:15:38.944643 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 15:15:38.944643 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 15:15:38.944643 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 15:15:38.944643 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 15:15:38.944643 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 15:15:38.944643 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:15:38.944643 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:38.944643 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 285/584] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:15:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:38.945148 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a150100-90c1-49a4-944c-d51c4e7b15e5 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:39.003831 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb50740-56db-4d2e-8f13-dfd91a147577 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:39.038676 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce01229e-b468-4222-b1ea-db3e323d5665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:39.039465 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01229e-b468-4222-b1ea-db3e323d5665 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:39.039776 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01229e-b468-4222-b1ea-db3e323d5665 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:39.040051 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01229e-b468-4222-b1ea-db3e323d5665 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:39.050715 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce01229e-b468-4222-b1ea-db3e323d5665 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:39.052205 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce01229e-b468-4222-b1ea-db3e323d5665 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:39.070272 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-ce01229e-b468-4222-b1ea-db3e323d5665 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:39.070272 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:39.070272 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:39.070272 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:39.070272 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:39.070272 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:39.070272 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:39.070272 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:39.070272 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:39.070272 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:39.070272 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:39.070272 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:39.070272 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:39.070272 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:39.070272 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:39.071424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:39.071424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:39.071424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:39.071424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:39.071424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:39.071424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:39.071424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:39.071424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:39.071424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:39.071424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:39.071424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:39.071424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:39.071424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:39.071424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:39.071424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:39.071424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:39.071424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:39.072655 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:39.072655 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:39.072655 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:39.072655 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:39.072655 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 300/585] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:39.076530 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce01229e-b468-4222-b1ea-db3e323d5665 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:39.079100 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a150100-90c1-49a4-944c-d51c4e7b15e5 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:39.113766 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea5df38-7f65-4c3a-9f87-7e503632dc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:39.114502 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea5df38-7f65-4c3a-9f87-7e503632dc8a None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:39.114850 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea5df38-7f65-4c3a-9f87-7e503632dc8a None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:39.115197 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea5df38-7f65-4c3a-9f87-7e503632dc8a None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:39.116053 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea5df38-7f65-4c3a-9f87-7e503632dc8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:39.116413 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea5df38-7f65-4c3a-9f87-7e503632dc8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:39.117851 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea5df38-7f65-4c3a-9f87-7e503632dc8a None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:39.125452 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:39.136215 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 286/586] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:39.136215 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea5df38-7f65-4c3a-9f87-7e503632dc8a None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:39.149541 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:39.182737 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce01229e-b468-4222-b1ea-db3e323d5665 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:39.225491 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0daaa99-6f59-425d-9b3e-f91b0efc6074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:39.226460 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0daaa99-6f59-425d-9b3e-f91b0efc6074 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:39.226856 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0daaa99-6f59-425d-9b3e-f91b0efc6074 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:39.227132 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0daaa99-6f59-425d-9b3e-f91b0efc6074 None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:39.227888 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0daaa99-6f59-425d-9b3e-f91b0efc6074 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:39.228194 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0daaa99-6f59-425d-9b3e-f91b0efc6074 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:39.229558 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0daaa99-6f59-425d-9b3e-f91b0efc6074 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:39.238153 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea5df38-7f65-4c3a-9f87-7e503632dc8a None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:39.245330 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 301/587] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:39.246144 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0daaa99-6f59-425d-9b3e-f91b0efc6074 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:39.277821 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73cbede-6d1a-4463-80d4-1b957d4a9d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:39.278771 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73cbede-6d1a-4463-80d4-1b957d4a9d83 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:39.279144 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73cbede-6d1a-4463-80d4-1b957d4a9d83 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:39.279425 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73cbede-6d1a-4463-80d4-1b957d4a9d83 None admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:39.280203 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73cbede-6d1a-4463-80d4-1b957d4a9d83 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:39.280513 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73cbede-6d1a-4463-80d4-1b957d4a9d83 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:39.282477 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73cbede-6d1a-4463-80d4-1b957d4a9d83 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:39.299397 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 287/588] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Jul 7 15:15:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:39.299397 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73cbede-6d1a-4463-80d4-1b957d4a9d83 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:39.355935 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0daaa99-6f59-425d-9b3e-f91b0efc6074 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:39.365822 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:39.382501 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:39.391551 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bf3fcc-7a7a-4040-b96b-8f1dba73b399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:39.392236 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf3fcc-7a7a-4040-b96b-8f1dba73b399 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:39.392554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf3fcc-7a7a-4040-b96b-8f1dba73b399 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:39.392827 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf3fcc-7a7a-4040-b96b-8f1dba73b399 None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:39.393758 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bf3fcc-7a7a-4040-b96b-8f1dba73b399 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:39.393882 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bf3fcc-7a7a-4040-b96b-8f1dba73b399 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:39.395204 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bf3fcc-7a7a-4040-b96b-8f1dba73b399 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:39.408389 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73cbede-6d1a-4463-80d4-1b957d4a9d83 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:39.413333 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 302/589] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:39.414093 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bf3fcc-7a7a-4040-b96b-8f1dba73b399 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:39.443969 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043e5cf-4526-4435-80cc-7d4f39ba6b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:39.444752 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043e5cf-4526-4435-80cc-7d4f39ba6b6e None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:39.445032 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043e5cf-4526-4435-80cc-7d4f39ba6b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:39.445305 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043e5cf-4526-4435-80cc-7d4f39ba6b6e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:39.453516 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f043e5cf-4526-4435-80cc-7d4f39ba6b6e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:39.454928 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f043e5cf-4526-4435-80cc-7d4f39ba6b6e None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:39.469746 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-f043e5cf-4526-4435-80cc-7d4f39ba6b6e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 07 15:15:39.469746 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:39.469746 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:39.469746 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:39.469746 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:39.469746 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:39.469746 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:39.469746 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:39.469746 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:39.469746 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:39.469746 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:39.469746 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:39.469746 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 15:15:39.469746 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 15:15:39.469746 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 15:15:39.469746 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 15:15:39.470803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:39.470803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:39.470803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 15:15:39.470803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:15:39.470803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 15:15:39.470803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:15:39.470803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:39.470803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:39.470803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:39.470803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:39.470803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:39.470803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:39.470803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:39.470803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:39.470803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:39.470803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:39.470803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:39.470803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:39.471811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:39.471811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:39.471811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 15:15:39.471811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 15:15:39.471811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 15:15:39.471811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 15:15:39.471811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 15:15:39.471811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 15:15:39.471811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 07 15:15:39.471811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:39.471811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 288/590] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:15:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:39.472411 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043e5cf-4526-4435-80cc-7d4f39ba6b6e None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:39.528179 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bf3fcc-7a7a-4040-b96b-8f1dba73b399 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:39.563689 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a01c43d-3e46-4c67-b6cb-94f9ef207f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:39.564483 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a01c43d-3e46-4c67-b6cb-94f9ef207f9f None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:39.564764 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a01c43d-3e46-4c67-b6cb-94f9ef207f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:39.565037 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a01c43d-3e46-4c67-b6cb-94f9ef207f9f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:39.573051 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a01c43d-3e46-4c67-b6cb-94f9ef207f9f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:39.574525 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a01c43d-3e46-4c67-b6cb-94f9ef207f9f None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:39.581052 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043e5cf-4526-4435-80cc-7d4f39ba6b6e None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:39.589940 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-3a01c43d-3e46-4c67-b6cb-94f9ef207f9f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 07 15:15:39.589940 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:39.589940 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:39.589940 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:39.589940 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:39.589940 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:39.589940 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:39.589940 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:39.589940 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:39.589940 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:39.589940 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:39.589940 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:39.589940 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 15:15:39.589940 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 15:15:39.589940 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 15:15:39.589940 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 15:15:39.591053 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:39.591053 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:39.591053 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 15:15:39.591053 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:15:39.591053 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 15:15:39.591053 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:15:39.591053 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:39.591053 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:39.591053 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:39.591053 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:39.591053 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:39.591053 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:39.591053 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:39.591053 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:39.591053 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:39.591053 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:39.591053 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:39.591053 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:39.592076 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:39.592076 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:39.592076 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 15:15:39.592076 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 15:15:39.592076 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 15:15:39.592076 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 15:15:39.592076 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 15:15:39.592076 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 15:15:39.592076 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 07 15:15:39.592076 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:39.592076 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 303/591] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Jul 7 15:15:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:39.592675 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a01c43d-3e46-4c67-b6cb-94f9ef207f9f None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:39.621436 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:39.621605 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c564b-f8af-4079-8faf-7ad11881b93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:39.622656 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c564b-f8af-4079-8faf-7ad11881b93d None admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:39.622783 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c564b-f8af-4079-8faf-7ad11881b93d None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:39.623048 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c564b-f8af-4079-8faf-7ad11881b93d None admin] PATH_INFO: `/v3/projects/c5260c7fd3614e9e93b167c86a94260e/users/de1632555cb345f2b9213da1925c02f4/roles/3d4f4e0a472e4ff9ac68d90c8612e956` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:39.626340 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:39.638582 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c564b-f8af-4079-8faf-7ad11881b93d None admin] RBAC: Authorizing `identity:create_grant(project_id=c5260c7fd3614e9e93b167c86a94260e, user_id=de1632555cb345f2b9213da1925c02f4, role_id=3d4f4e0a472e4ff9ac68d90c8612e956)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:39.640897 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c564b-f8af-4079-8faf-7ad11881b93d None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:39.693840 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a01c43d-3e46-4c67-b6cb-94f9ef207f9f None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:39.718419 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 289/592] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Jul 7 15:15:39 2022] PUT /identity/v3/projects/c5260c7fd3614e9e93b167c86a94260e/users/de1632555cb345f2b9213da1925c02f4/roles/3d4f4e0a472e4ff9ac68d90c8612e956 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:15:39.719443 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c564b-f8af-4079-8faf-7ad11881b93d None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:39.740329 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad231e-d928-456a-a75f-4496d7f4042b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:39.741509 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad231e-d928-456a-a75f-4496d7f4042b None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:39.742031 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad231e-d928-456a-a75f-4496d7f4042b None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:39.742573 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad231e-d928-456a-a75f-4496d7f4042b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:39.751057 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad231e-d928-456a-a75f-4496d7f4042b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:39.752980 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad231e-d928-456a-a75f-4496d7f4042b None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:39.769208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-cfad231e-d928-456a-a75f-4496d7f4042b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:15:39.769208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:39.769208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:39.769208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:39.769208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:39.769208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:39.769208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:39.769208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:39.769208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:39.769208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:39.769208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:39.769208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:39.769208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 15:15:39.769208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 15:15:39.769208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 15:15:39.770180 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 15:15:39.770180 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:39.770180 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:39.770180 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:39.770180 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:39.770180 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:39.770180 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:39.770180 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:39.770180 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:39.770180 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:39.770180 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:39.770180 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:39.770180 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:39.770180 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:39.770180 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:39.770180 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:39.770180 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:39.771076 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 15:15:39.771076 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 15:15:39.771076 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 15:15:39.771076 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 15:15:39.771076 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 15:15:39.771076 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 15:15:39.771076 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:15:39.771076 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:39.773399 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 304/593] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:15:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:39.774500 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad231e-d928-456a-a75f-4496d7f4042b None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:39.884512 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:39.886327 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c564b-f8af-4079-8faf-7ad11881b93d None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:39.900852 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad231e-d928-456a-a75f-4496d7f4042b None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:39.926265 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e3bf22-fcaf-48a8-9715-3db5d66155a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:39.926939 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3bf22-fcaf-48a8-9715-3db5d66155a3 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:39.927227 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3bf22-fcaf-48a8-9715-3db5d66155a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:39.927509 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3bf22-fcaf-48a8-9715-3db5d66155a3 None admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:39.928442 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e3bf22-fcaf-48a8-9715-3db5d66155a3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:39.928804 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e3bf22-fcaf-48a8-9715-3db5d66155a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:39.930527 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e3bf22-fcaf-48a8-9715-3db5d66155a3 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:39.938751 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618599c4-815b-4720-be10-069b7d527c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:39.939423 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618599c4-815b-4720-be10-069b7d527c9a None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:39.939790 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618599c4-815b-4720-be10-069b7d527c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:39.940065 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618599c4-815b-4720-be10-069b7d527c9a None admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:39.940905 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618599c4-815b-4720-be10-069b7d527c9a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:39.941211 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618599c4-815b-4720-be10-069b7d527c9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:39.942610 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618599c4-815b-4720-be10-069b7d527c9a None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:39.949607 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 290/594] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Jul 7 15:15:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:39.950267 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e3bf22-fcaf-48a8-9715-3db5d66155a3 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:39.966258 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 305/595] 10.222.0.25 () {56 vars in 1139 bytes} [Thu Jul 7 15:15:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:39.967907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c38318-d84a-4c94-9fe5-6ebfc0453698 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:39.968231 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c38318-d84a-4c94-9fe5-6ebfc0453698 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:39.968510 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c38318-d84a-4c94-9fe5-6ebfc0453698 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:39.969866 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 306/596] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:39.973082 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c38318-d84a-4c94-9fe5-6ebfc0453698 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:40.004435 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:40.063897 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e3bf22-fcaf-48a8-9715-3db5d66155a3 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:40.090966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c38318-d84a-4c94-9fe5-6ebfc0453698 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:40.098761 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963fef6b-30dc-483a-abf8-33ad30b55318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:40.099498 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963fef6b-30dc-483a-abf8-33ad30b55318 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:40.099876 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963fef6b-30dc-483a-abf8-33ad30b55318 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:40.100225 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963fef6b-30dc-483a-abf8-33ad30b55318 None admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:40.101093 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963fef6b-30dc-483a-abf8-33ad30b55318 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:40.101473 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963fef6b-30dc-483a-abf8-33ad30b55318 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:40.102948 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963fef6b-30dc-483a-abf8-33ad30b55318 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:40.115372 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 291/597] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Jul 7 15:15:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:40.116295 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963fef6b-30dc-483a-abf8-33ad30b55318 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:40.136666 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959b1bcc-ca07-4f6b-938f-f88de32f8d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:40.137386 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959b1bcc-ca07-4f6b-938f-f88de32f8d5d None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:40.137667 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959b1bcc-ca07-4f6b-938f-f88de32f8d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:40.137960 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959b1bcc-ca07-4f6b-938f-f88de32f8d5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:40.148618 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959b1bcc-ca07-4f6b-938f-f88de32f8d5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:40.150626 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959b1bcc-ca07-4f6b-938f-f88de32f8d5d None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:40.158442 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:40.169614 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-959b1bcc-ca07-4f6b-938f-f88de32f8d5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:40.169614 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:40.169614 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:40.169614 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:40.169614 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:40.169614 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:40.169614 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:40.169614 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:40.169614 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:40.169614 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:40.169614 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:40.169614 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:40.169614 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:40.169614 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:40.169614 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:40.170554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:40.170554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:40.170554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:40.170554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:40.170554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:40.170554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:40.170554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:40.170554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:40.170554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:40.170554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:40.170554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:40.170554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:40.170554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:40.170554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:40.170554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:40.170554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:40.170554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:40.172087 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:40.172087 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:40.172087 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:40.172087 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:40.172087 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 307/598] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:40.172963 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e2a55-81a8-423e-9b51-1a3fea92c997 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:40.173241 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e2a55-81a8-423e-9b51-1a3fea92c997 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:40.173511 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e2a55-81a8-423e-9b51-1a3fea92c997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:40.190744 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-9c1e2a55-81a8-423e-9b51-1a3fea92c997 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:40.191951 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-9c1e2a55-81a8-423e-9b51-1a3fea92c997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:40.234770 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963fef6b-30dc-483a-abf8-33ad30b55318 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:40.236638 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:40.271791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 308/599] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:40.271791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e2a55-81a8-423e-9b51-1a3fea92c997 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:40.275521 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b50cd9f-9bb4-4948-a3c3-8ca84f44f00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:40.276672 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50cd9f-9bb4-4948-a3c3-8ca84f44f00c None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:40.277686 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50cd9f-9bb4-4948-a3c3-8ca84f44f00c None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:40.278188 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50cd9f-9bb4-4948-a3c3-8ca84f44f00c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:40.293120 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b50cd9f-9bb4-4948-a3c3-8ca84f44f00c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:40.295057 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b50cd9f-9bb4-4948-a3c3-8ca84f44f00c None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:40.315765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-7b50cd9f-9bb4-4948-a3c3-8ca84f44f00c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:40.315765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:40.315765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:40.315765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:40.315765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:40.315765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:40.315765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:40.315765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:40.315765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:40.315765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:40.315765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:40.315765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:40.315765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:40.315765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:40.315765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:40.316863 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:40.316863 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:40.316863 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:40.316863 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:40.316863 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:40.316863 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:40.316863 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:40.316863 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:40.316863 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:40.316863 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:40.316863 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:40.316863 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:40.316863 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:40.316863 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:40.316863 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:40.316863 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:40.316863 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:40.318000 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:40.318000 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:40.318000 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:40.318000 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:40.321600 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 292/600] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:40.322419 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b50cd9f-9bb4-4948-a3c3-8ca84f44f00c None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:40.376385 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e2a55-81a8-423e-9b51-1a3fea92c997 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:40.405488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:40.414144 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32439b2e-4fe2-4b7d-a5ee-9a51f5615eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:40.421202 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32439b2e-4fe2-4b7d-a5ee-9a51f5615eb7 None admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:40.421202 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32439b2e-4fe2-4b7d-a5ee-9a51f5615eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:40.421202 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32439b2e-4fe2-4b7d-a5ee-9a51f5615eb7 None admin] PATH_INFO: `/v3/projects/c5260c7fd3614e9e93b167c86a94260e/users/3d0d29e718f446e2a9fd1b2575a86041/roles/3d4f4e0a472e4ff9ac68d90c8612e956` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:40.428952 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b50cd9f-9bb4-4948-a3c3-8ca84f44f00c None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:40.430896 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32439b2e-4fe2-4b7d-a5ee-9a51f5615eb7 None admin] RBAC: Authorizing `identity:create_grant(project_id=c5260c7fd3614e9e93b167c86a94260e, user_id=3d0d29e718f446e2a9fd1b2575a86041, role_id=3d4f4e0a472e4ff9ac68d90c8612e956)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:40.432490 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32439b2e-4fe2-4b7d-a5ee-9a51f5615eb7 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:40.464621 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c7c93-eff0-47fa-a4e7-75aecd85d354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:40.465315 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c7c93-eff0-47fa-a4e7-75aecd85d354 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:40.465554 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c7c93-eff0-47fa-a4e7-75aecd85d354 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:40.465825 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c7c93-eff0-47fa-a4e7-75aecd85d354 None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:40.466541 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622c7c93-eff0-47fa-a4e7-75aecd85d354 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:40.466872 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622c7c93-eff0-47fa-a4e7-75aecd85d354 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:40.468840 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622c7c93-eff0-47fa-a4e7-75aecd85d354 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:40.480373 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 293/601] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:40.481142 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c7c93-eff0-47fa-a4e7-75aecd85d354 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:40.499423 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:40.548580 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 309/602] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Jul 7 15:15:40 2022] PUT /identity/v3/projects/c5260c7fd3614e9e93b167c86a94260e/users/3d0d29e718f446e2a9fd1b2575a86041/roles/3d4f4e0a472e4ff9ac68d90c8612e956 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:15:40.548580 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32439b2e-4fe2-4b7d-a5ee-9a51f5615eb7 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:40.602931 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c7c93-eff0-47fa-a4e7-75aecd85d354 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:40.642903 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e897922-4f4c-4f16-9c4d-97145b76f8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:40.643927 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e897922-4f4c-4f16-9c4d-97145b76f8ad None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:40.644310 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e897922-4f4c-4f16-9c4d-97145b76f8ad None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:40.644691 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e897922-4f4c-4f16-9c4d-97145b76f8ad None admin] PATH_INFO: `/v3/roles/service` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:40.657192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e897922-4f4c-4f16-9c4d-97145b76f8ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:40.665253 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-5e897922-4f4c-4f16-9c4d-97145b76f8ad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:15:40.665253 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:40.665253 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:40.665253 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:40.665253 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:40.665253 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:40.665253 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:40.665253 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:40.665253 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:40.665253 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:40.665253 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:40.665253 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:40.665253 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:40.665253 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:40.665253 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:40.665253 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:40.666461 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:40.666461 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:40.666461 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:40.666461 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:40.666461 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:40.666461 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:40.666461 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:40.666461 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:40.666461 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:40.666461 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:40.666461 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:40.666461 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:40.666461 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:40.666461 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:40.666461 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:40.666461 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:40.667546 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:40.667546 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:40.667546 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:40.667546 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:40.667546 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:40.667546 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:40.667546 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:40.667546 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:40.667546 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:40.667546 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:40.667546 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:15:40.667546 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:40.674335 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 294/603] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Jul 7 15:15:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:40.674335 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e897922-4f4c-4f16-9c4d-97145b76f8ad None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:40.684372 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32439b2e-4fe2-4b7d-a5ee-9a51f5615eb7 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:40.738421 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5a749-484d-4dcb-a50d-6302a735af56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:40.739259 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5a749-484d-4dcb-a50d-6302a735af56 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:40.739690 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5a749-484d-4dcb-a50d-6302a735af56 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:40.740073 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5a749-484d-4dcb-a50d-6302a735af56 None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:40.740911 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5a749-484d-4dcb-a50d-6302a735af56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:40.741298 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5a749-484d-4dcb-a50d-6302a735af56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:40.742736 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5a749-484d-4dcb-a50d-6302a735af56 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:40.758659 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 310/604] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:40.759842 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5a749-484d-4dcb-a50d-6302a735af56 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:40.773272 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:40.800436 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e897922-4f4c-4f16-9c4d-97145b76f8ad None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:40.811789 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:40.837624 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3d0dc-64db-4c79-9129-1381eca46f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:40.838659 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3d0dc-64db-4c79-9129-1381eca46f58 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:40.839088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3d0dc-64db-4c79-9129-1381eca46f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:40.839540 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3d0dc-64db-4c79-9129-1381eca46f58 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:40.847601 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3d0dc-64db-4c79-9129-1381eca46f58 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:40.849527 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3d0dc-64db-4c79-9129-1381eca46f58 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:40.865336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-c0e3d0dc-64db-4c79-9129-1381eca46f58 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:15:40.865336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:40.865336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:40.865336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:40.865336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:40.865336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:40.865336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:40.865336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:40.865336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:40.865336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:40.865336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:40.865336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:40.865336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 15:15:40.865336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 15:15:40.865336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 15:15:40.866633 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 15:15:40.866633 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:40.866633 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:40.866633 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:40.866633 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:40.866633 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:40.866633 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:40.866633 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:40.866633 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:40.866633 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:40.866633 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:40.866633 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:40.866633 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:40.866633 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:40.866633 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:40.866633 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:40.866633 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:40.867774 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 15:15:40.867774 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 15:15:40.867774 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 15:15:40.867774 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 15:15:40.867774 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 15:15:40.867774 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 15:15:40.867774 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:15:40.867774 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:40.870732 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 295/605] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:15:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:40.871867 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3d0dc-64db-4c79-9129-1381eca46f58 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:40.881350 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5a749-484d-4dcb-a50d-6302a735af56 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:40.918310 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a8d25-63b9-4c59-90d3-7d91537544ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:40.918650 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a8d25-63b9-4c59-90d3-7d91537544ad None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:40.918934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a8d25-63b9-4c59-90d3-7d91537544ad None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:40.919208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a8d25-63b9-4c59-90d3-7d91537544ad None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:40.920056 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6a8d25-63b9-4c59-90d3-7d91537544ad None admin] RBAC: Adding query filter params (name=service) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:40.920362 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6a8d25-63b9-4c59-90d3-7d91537544ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:40.921763 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6a8d25-63b9-4c59-90d3-7d91537544ad None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:40.930594 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 311/606] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Jul 7 15:15:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:40.931507 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a8d25-63b9-4c59-90d3-7d91537544ad None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:40.987154 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3d0dc-64db-4c79-9129-1381eca46f58 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:41.024226 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d590db53-4949-40f1-924c-2705cb34228e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:41.025260 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590db53-4949-40f1-924c-2705cb34228e None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:41.025691 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590db53-4949-40f1-924c-2705cb34228e None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:41.026280 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590db53-4949-40f1-924c-2705cb34228e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:41.028447 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:41.034694 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d590db53-4949-40f1-924c-2705cb34228e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:41.034838 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:41.036542 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d590db53-4949-40f1-924c-2705cb34228e None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:41.038099 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a8d25-63b9-4c59-90d3-7d91537544ad None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:41.050874 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-d590db53-4949-40f1-924c-2705cb34228e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:15:41.050874 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:41.050874 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:41.050874 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:41.050874 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:41.050874 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:41.050874 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:41.050874 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:41.050874 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:41.050874 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:41.050874 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:41.050874 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:41.050874 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 15:15:41.050874 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 15:15:41.050874 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 15:15:41.054959 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 15:15:41.054959 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:41.054959 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:41.054959 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:41.054959 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:41.054959 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:41.054959 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:41.054959 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:41.054959 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:41.054959 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:41.054959 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:41.054959 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:41.054959 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:41.054959 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:41.054959 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:41.054959 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:41.054959 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:41.055982 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 15:15:41.055982 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 15:15:41.055982 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 15:15:41.055982 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 15:15:41.055982 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 15:15:41.055982 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 15:15:41.055982 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:15:41.055982 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:41.055982 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 296/607] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:15:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:41.055982 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d590db53-4949-40f1-924c-2705cb34228e None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:41.072392 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51ad22-707a-4b1b-9f45-f713c6beb845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:41.073057 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51ad22-707a-4b1b-9f45-f713c6beb845 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:41.073335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51ad22-707a-4b1b-9f45-f713c6beb845 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:41.073608 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51ad22-707a-4b1b-9f45-f713c6beb845 None admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:41.074391 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d51ad22-707a-4b1b-9f45-f713c6beb845 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:41.075052 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d51ad22-707a-4b1b-9f45-f713c6beb845 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:41.076455 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d51ad22-707a-4b1b-9f45-f713c6beb845 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:41.090038 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 312/608] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Jul 7 15:15:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:41.090892 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51ad22-707a-4b1b-9f45-f713c6beb845 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:41.168038 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d590db53-4949-40f1-924c-2705cb34228e None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:41.194026 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51ad22-707a-4b1b-9f45-f713c6beb845 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:41.204466 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c021f-a330-4fd0-a8a9-3fe214e60d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:41.205427 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c021f-a330-4fd0-a8a9-3fe214e60d94 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:41.205905 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c021f-a330-4fd0-a8a9-3fe214e60d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:41.206347 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c021f-a330-4fd0-a8a9-3fe214e60d94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:41.217739 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c021f-a330-4fd0-a8a9-3fe214e60d94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:41.219384 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c021f-a330-4fd0-a8a9-3fe214e60d94 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:41.229971 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc7f78-6d01-446a-9194-8a560f3e897e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:41.230635 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc7f78-6d01-446a-9194-8a560f3e897e None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:41.230916 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc7f78-6d01-446a-9194-8a560f3e897e None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:41.231190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc7f78-6d01-446a-9194-8a560f3e897e None admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:41.231998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc7f78-6d01-446a-9194-8a560f3e897e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:41.232304 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc7f78-6d01-446a-9194-8a560f3e897e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:41.234243 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc7f78-6d01-446a-9194-8a560f3e897e None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:41.239640 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-f94c021f-a330-4fd0-a8a9-3fe214e60d94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:41.239640 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:41.239640 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:41.239640 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:41.239640 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:41.239640 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:41.239640 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:41.239640 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:41.239640 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:41.239640 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:41.239640 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:41.239640 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:41.239640 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:41.239640 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:41.239640 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:41.240708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:41.240708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:41.240708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:41.240708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:41.240708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:41.240708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:41.240708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:41.240708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:41.240708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:41.240708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:41.240708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:41.240708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:41.240708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:41.240708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:41.240708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:41.240708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:41.240708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:41.241723 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:41.241723 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:41.241723 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:41.241723 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:41.243635 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 297/609] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:41.244533 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c021f-a330-4fd0-a8a9-3fe214e60d94 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:41.246547 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 313/610] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Jul 7 15:15:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:41.256559 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc7f78-6d01-446a-9194-8a560f3e897e None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:41.285178 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:41.297497 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:41.360890 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c021f-a330-4fd0-a8a9-3fe214e60d94 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:41.383929 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc7f78-6d01-446a-9194-8a560f3e897e None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:41.396730 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266cf27d-55ac-43b1-b31b-5baf666d8ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:41.397773 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266cf27d-55ac-43b1-b31b-5baf666d8ceb None admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:41.398166 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266cf27d-55ac-43b1-b31b-5baf666d8ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:41.398553 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266cf27d-55ac-43b1-b31b-5baf666d8ceb None admin] PATH_INFO: `/v3/projects/c5260c7fd3614e9e93b167c86a94260e/users/2a0fbebb3dd2458ba4fe5d7f0ce211b8/roles/3d4f4e0a472e4ff9ac68d90c8612e956` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:41.413965 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266cf27d-55ac-43b1-b31b-5baf666d8ceb None admin] RBAC: Authorizing `identity:create_grant(project_id=c5260c7fd3614e9e93b167c86a94260e, user_id=2a0fbebb3dd2458ba4fe5d7f0ce211b8, role_id=3d4f4e0a472e4ff9ac68d90c8612e956)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:41.415619 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266cf27d-55ac-43b1-b31b-5baf666d8ceb None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:41.422522 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47855818-c4b5-449a-acaa-69478f73fd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:41.423239 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47855818-c4b5-449a-acaa-69478f73fd68 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:41.423528 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47855818-c4b5-449a-acaa-69478f73fd68 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:41.423848 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47855818-c4b5-449a-acaa-69478f73fd68 None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:41.424575 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47855818-c4b5-449a-acaa-69478f73fd68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:41.424902 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47855818-c4b5-449a-acaa-69478f73fd68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:41.426349 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47855818-c4b5-449a-acaa-69478f73fd68 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:41.443200 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 314/611] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:41.443200 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47855818-c4b5-449a-acaa-69478f73fd68 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:41.519821 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 298/612] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Jul 7 15:15:41 2022] PUT /identity/v3/projects/c5260c7fd3614e9e93b167c86a94260e/users/2a0fbebb3dd2458ba4fe5d7f0ce211b8/roles/3d4f4e0a472e4ff9ac68d90c8612e956 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:15:41.521152 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266cf27d-55ac-43b1-b31b-5baf666d8ceb None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:41.533921 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:41.557886 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47855818-c4b5-449a-acaa-69478f73fd68 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:41.601145 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4645d22-83d6-496f-a664-fc2eb00f01c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:41.602056 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4645d22-83d6-496f-a664-fc2eb00f01c7 None admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:41.602331 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4645d22-83d6-496f-a664-fc2eb00f01c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:41.602601 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4645d22-83d6-496f-a664-fc2eb00f01c7 None admin] PATH_INFO: `/v3/projects/c5260c7fd3614e9e93b167c86a94260e/users/e79efaea155c46ecb9831956e68aec8e/roles/3d4f4e0a472e4ff9ac68d90c8612e956` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:41.620505 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4645d22-83d6-496f-a664-fc2eb00f01c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=c5260c7fd3614e9e93b167c86a94260e, user_id=e79efaea155c46ecb9831956e68aec8e, role_id=3d4f4e0a472e4ff9ac68d90c8612e956)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:41.622623 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4645d22-83d6-496f-a664-fc2eb00f01c7 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:41.702259 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:41.709695 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266cf27d-55ac-43b1-b31b-5baf666d8ceb None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:41.718565 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 315/613] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Jul 7 15:15:41 2022] PUT /identity/v3/projects/c5260c7fd3614e9e93b167c86a94260e/users/e79efaea155c46ecb9831956e68aec8e/roles/3d4f4e0a472e4ff9ac68d90c8612e956 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:15:41.757211 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a2f4d-6197-4e7d-b6d5-f17eef989286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:41.758019 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a2f4d-6197-4e7d-b6d5-f17eef989286 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:41.758300 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a2f4d-6197-4e7d-b6d5-f17eef989286 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:41.758573 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a2f4d-6197-4e7d-b6d5-f17eef989286 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:41.767754 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112a2f4d-6197-4e7d-b6d5-f17eef989286 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:41.769336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112a2f4d-6197-4e7d-b6d5-f17eef989286 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:41.786749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-112a2f4d-6197-4e7d-b6d5-f17eef989286 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 07 15:15:41.786749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:41.786749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:41.786749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:41.786749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:41.786749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:41.786749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:41.786749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:41.786749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:41.786749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:41.786749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:41.786749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:41.786749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 15:15:41.786749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 15:15:41.786749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 15:15:41.786749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 15:15:41.787798 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:41.787798 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:41.787798 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 15:15:41.787798 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:15:41.787798 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 15:15:41.787798 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:15:41.787798 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:41.787798 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:41.787798 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:41.787798 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:41.787798 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:41.787798 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:41.787798 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:41.787798 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:41.787798 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:41.787798 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:41.787798 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:41.787798 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:41.788788 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:41.788788 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:41.788788 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 15:15:41.788788 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 15:15:41.788788 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 15:15:41.788788 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 15:15:41.788788 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 15:15:41.788788 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 15:15:41.788788 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 07 15:15:41.788788 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:41.788788 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 299/614] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Jul 7 15:15:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:41.796020 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4645d22-83d6-496f-a664-fc2eb00f01c7 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:41.826213 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd4706-90f3-4f3d-97ac-c72b45a9c8f3 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:41.826213 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd4706-90f3-4f3d-97ac-c72b45a9c8f3 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:41.826213 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd4706-90f3-4f3d-97ac-c72b45a9c8f3 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:41.826213 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 300/615] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:15:41.844138 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0f327-6f3d-468e-8899-f841c4b16032 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:41.844138 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0f327-6f3d-468e-8899-f841c4b16032 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:41.844138 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0f327-6f3d-468e-8899-f841c4b16032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:41.862434 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-3dd0f327-6f3d-468e-8899-f841c4b16032 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:41.863844 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-3dd0f327-6f3d-468e-8899-f841c4b16032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:41.938686 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4645d22-83d6-496f-a664-fc2eb00f01c7 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:41.957199 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 301/616] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 15:15:41.977664 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:41.981658 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd0f327-6f3d-468e-8899-f841c4b16032 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:41.991759 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ded232-37da-4fb5-a6ff-bb2e7af4e462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:41.992896 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ded232-37da-4fb5-a6ff-bb2e7af4e462 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:41.993340 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ded232-37da-4fb5-a6ff-bb2e7af4e462 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:41.993776 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ded232-37da-4fb5-a6ff-bb2e7af4e462 None admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:41.995157 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ded232-37da-4fb5-a6ff-bb2e7af4e462 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:41.995698 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ded232-37da-4fb5-a6ff-bb2e7af4e462 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:41.997489 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ded232-37da-4fb5-a6ff-bb2e7af4e462 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:42.018540 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 316/617] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Jul 7 15:15:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:42.025382 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ded232-37da-4fb5-a6ff-bb2e7af4e462 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:42.091256 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:42.107014 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd0f327-6f3d-468e-8899-f841c4b16032 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:42.141970 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ded232-37da-4fb5-a6ff-bb2e7af4e462 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:42.144188 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c3c317-c3b0-4256-9fbe-44c6dfd62b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:42.145147 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3c317-c3b0-4256-9fbe-44c6dfd62b7a None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:42.145595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3c317-c3b0-4256-9fbe-44c6dfd62b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:42.146016 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3c317-c3b0-4256-9fbe-44c6dfd62b7a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:42.157071 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c3c317-c3b0-4256-9fbe-44c6dfd62b7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:42.165032 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-48c3c317-c3b0-4256-9fbe-44c6dfd62b7a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:15:42.165032 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:42.165032 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:42.165032 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:42.165032 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:42.165032 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:42.165032 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:42.165032 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:42.165032 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:42.165032 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:42.165032 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:42.165032 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:42.165032 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:42.165032 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:42.165032 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:42.165032 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:42.166060 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:42.166060 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:42.166060 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:42.166060 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:42.166060 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:42.166060 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:42.166060 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:42.166060 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:42.166060 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:42.166060 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:42.166060 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:42.166060 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:42.166060 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:42.166060 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:42.166060 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:42.166060 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:42.167080 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:42.167080 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:42.167080 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:42.167080 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:42.167080 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:42.167080 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:42.167080 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:42.167080 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:42.167080 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:42.167080 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:42.167080 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:15:42.167080 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:42.169680 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 302/618] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Jul 7 15:15:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:42.183842 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c3c317-c3b0-4256-9fbe-44c6dfd62b7a None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:42.186413 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42791ee-167d-4920-bc9b-1249810d34a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:42.187965 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42791ee-167d-4920-bc9b-1249810d34a2 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:42.188318 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42791ee-167d-4920-bc9b-1249810d34a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:42.188694 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42791ee-167d-4920-bc9b-1249810d34a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:42.203589 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42791ee-167d-4920-bc9b-1249810d34a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:42.203589 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42791ee-167d-4920-bc9b-1249810d34a2 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:42.225119 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-e42791ee-167d-4920-bc9b-1249810d34a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:42.225119 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:42.225119 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:42.225119 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:42.225119 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:42.225119 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:42.225119 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:42.225119 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:42.225119 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:42.225119 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:42.225119 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:42.225119 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:42.225119 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:42.225119 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:42.225119 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:42.226287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:42.226287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:42.226287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:42.226287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:42.226287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:42.226287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:42.226287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:42.226287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:42.226287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:42.226287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:42.226287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:42.226287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:42.226287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:42.226287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:42.226287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:42.226287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:42.226287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:42.227385 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:42.227385 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:42.227385 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:42.227385 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:42.230171 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 317/619] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:42.237624 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42791ee-167d-4920-bc9b-1249810d34a2 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:42.318206 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:42.350101 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c3c317-c3b0-4256-9fbe-44c6dfd62b7a None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:42.394268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d209cc-143d-4419-926c-b4460246ac90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:42.396130 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d209cc-143d-4419-926c-b4460246ac90 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:42.399308 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d209cc-143d-4419-926c-b4460246ac90 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:42.399831 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d209cc-143d-4419-926c-b4460246ac90 None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:42.400799 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d209cc-143d-4419-926c-b4460246ac90 None admin] RBAC: Adding query filter params (name=service) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:42.401316 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d209cc-143d-4419-926c-b4460246ac90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:42.402895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d209cc-143d-4419-926c-b4460246ac90 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:42.418286 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 303/620] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Jul 7 15:15:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:42.425024 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d209cc-143d-4419-926c-b4460246ac90 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:42.429451 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42791ee-167d-4920-bc9b-1249810d34a2 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:42.446149 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:42.466671 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f525f401-7fc8-4be5-b455-b4f0d2c694b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:42.478004 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f525f401-7fc8-4be5-b455-b4f0d2c694b6 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:42.478004 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f525f401-7fc8-4be5-b455-b4f0d2c694b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:42.478004 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f525f401-7fc8-4be5-b455-b4f0d2c694b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:42.478004 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f525f401-7fc8-4be5-b455-b4f0d2c694b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:42.478004 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f525f401-7fc8-4be5-b455-b4f0d2c694b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:42.478004 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f525f401-7fc8-4be5-b455-b4f0d2c694b6 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:42.487624 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 318/621] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:15:42.493934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f525f401-7fc8-4be5-b455-b4f0d2c694b6 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:42.609336 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:42.613822 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f525f401-7fc8-4be5-b455-b4f0d2c694b6 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:42.657222 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987eef63-c4ac-4dc8-b2e1-beff0660aa10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:42.658689 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987eef63-c4ac-4dc8-b2e1-beff0660aa10 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:42.659135 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987eef63-c4ac-4dc8-b2e1-beff0660aa10 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:42.659587 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987eef63-c4ac-4dc8-b2e1-beff0660aa10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:42.675027 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987eef63-c4ac-4dc8-b2e1-beff0660aa10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:42.677071 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987eef63-c4ac-4dc8-b2e1-beff0660aa10 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:42.693558 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d209cc-143d-4419-926c-b4460246ac90 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:42.694166 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-987eef63-c4ac-4dc8-b2e1-beff0660aa10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:15:42.694166 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:42.694166 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:42.694166 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:42.694166 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:42.694166 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:42.694166 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:42.694166 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:42.694166 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:42.694166 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:42.694166 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:42.694166 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:42.694166 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 15:15:42.694166 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 15:15:42.694166 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 15:15:42.695293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 15:15:42.695293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:42.695293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:42.695293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:42.695293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:42.695293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:42.695293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:42.695293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:42.695293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:42.695293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:42.695293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:42.695293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:42.695293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:42.695293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:42.695293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:42.695293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:42.695293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:42.696394 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 15:15:42.696394 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 15:15:42.696394 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 15:15:42.696394 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 15:15:42.696394 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 15:15:42.696394 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 15:15:42.696394 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:15:42.696394 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:42.699409 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 319/622] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:15:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:42.705632 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987eef63-c4ac-4dc8-b2e1-beff0660aa10 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:42.792291 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945097de-2f52-427d-9825-d1346890ca06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:42.793107 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945097de-2f52-427d-9825-d1346890ca06 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:42.793397 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945097de-2f52-427d-9825-d1346890ca06 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:42.793679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945097de-2f52-427d-9825-d1346890ca06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:42.814889 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987eef63-c4ac-4dc8-b2e1-beff0660aa10 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:42.820628 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945097de-2f52-427d-9825-d1346890ca06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:42.822086 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945097de-2f52-427d-9825-d1346890ca06 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:42.853005 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab28488-93ed-439a-a86e-7980e765bd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:42.853888 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab28488-93ed-439a-a86e-7980e765bd2c None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:42.854343 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab28488-93ed-439a-a86e-7980e765bd2c None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:42.854786 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab28488-93ed-439a-a86e-7980e765bd2c None admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:42.861229 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab28488-93ed-439a-a86e-7980e765bd2c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:42.861767 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab28488-93ed-439a-a86e-7980e765bd2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:42.863353 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab28488-93ed-439a-a86e-7980e765bd2c None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:42.863625 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-945097de-2f52-427d-9825-d1346890ca06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:42.863625 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:42.863625 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:42.863625 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:42.863625 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:42.863625 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:42.863625 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:42.863625 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:42.863625 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:42.863625 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:42.863625 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:42.863625 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:42.863625 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:42.863625 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:42.863625 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:42.864690 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:42.864690 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:42.864690 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:42.864690 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:42.864690 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:42.864690 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:42.864690 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:42.864690 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:42.864690 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:42.864690 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:42.864690 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:42.864690 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:42.864690 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:42.864690 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:42.864690 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:42.864690 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:42.864690 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:42.865799 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:42.865799 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:42.865799 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:42.865799 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:42.865799 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 304/623] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 440 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:42.876498 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945097de-2f52-427d-9825-d1346890ca06 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:42.883271 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 320/624] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Jul 7 15:15:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:15:42.889493 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab28488-93ed-439a-a86e-7980e765bd2c None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:42.935856 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:42.973804 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:43.034806 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945097de-2f52-427d-9825-d1346890ca06 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:43.070320 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab28488-93ed-439a-a86e-7980e765bd2c None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:43.075268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d598be2c-2a8f-471e-9219-ea3c6f70cd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:43.075983 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598be2c-2a8f-471e-9219-ea3c6f70cd5a None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:43.076265 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598be2c-2a8f-471e-9219-ea3c6f70cd5a None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:43.076541 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598be2c-2a8f-471e-9219-ea3c6f70cd5a None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:43.077284 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d598be2c-2a8f-471e-9219-ea3c6f70cd5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:43.078146 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d598be2c-2a8f-471e-9219-ea3c6f70cd5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:43.079517 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d598be2c-2a8f-471e-9219-ea3c6f70cd5a None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:43.098208 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 305/625] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:43.100545 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39867bc5-484e-412b-b55f-73f0570b5ae6 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:43.100545 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39867bc5-484e-412b-b55f-73f0570b5ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:43.100545 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39867bc5-484e-412b-b55f-73f0570b5ae6 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:43.103051 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 306/626] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:43.103965 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39867bc5-484e-412b-b55f-73f0570b5ae6 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:43.114665 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7f4773-d444-44d3-8db4-1a3c28c464ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:43.115971 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f4773-d444-44d3-8db4-1a3c28c464ab None admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:43.116251 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f4773-d444-44d3-8db4-1a3c28c464ab None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:43.116524 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f4773-d444-44d3-8db4-1a3c28c464ab None admin] PATH_INFO: `/v3/projects/c5260c7fd3614e9e93b167c86a94260e/users/b18033f849b240999d12c1885d2f7b91/roles/3d4f4e0a472e4ff9ac68d90c8612e956` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:43.138108 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7f4773-d444-44d3-8db4-1a3c28c464ab None admin] RBAC: Authorizing `identity:create_grant(project_id=c5260c7fd3614e9e93b167c86a94260e, user_id=b18033f849b240999d12c1885d2f7b91, role_id=3d4f4e0a472e4ff9ac68d90c8612e956)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:43.140581 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7f4773-d444-44d3-8db4-1a3c28c464ab None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:43.215082 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39867bc5-484e-412b-b55f-73f0570b5ae6 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:43.237543 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:43.247835 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 321/627] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Jul 7 15:15:42 2022] PUT /identity/v3/projects/c5260c7fd3614e9e93b167c86a94260e/users/b18033f849b240999d12c1885d2f7b91/roles/3d4f4e0a472e4ff9ac68d90c8612e956 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:15:43.250535 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c668d-21ab-4bc6-b4eb-4373684609d7 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:43.251453 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c668d-21ab-4bc6-b4eb-4373684609d7 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:43.252002 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c668d-21ab-4bc6-b4eb-4373684609d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:43.269417 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdbaee4-73d9-46f1-b314-0b94929a89a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:43.271957 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdbaee4-73d9-46f1-b314-0b94929a89a6 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:43.271957 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdbaee4-73d9-46f1-b314-0b94929a89a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:43.271957 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdbaee4-73d9-46f1-b314-0b94929a89a6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:43.289789 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-422c668d-21ab-4bc6-b4eb-4373684609d7 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:43.291461 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdbaee4-73d9-46f1-b314-0b94929a89a6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:43.293129 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdbaee4-73d9-46f1-b314-0b94929a89a6 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:43.294229 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-422c668d-21ab-4bc6-b4eb-4373684609d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:43.331527 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-fbdbaee4-73d9-46f1-b314-0b94929a89a6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 07 15:15:43.331527 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:43.331527 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:43.331527 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:43.331527 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:43.331527 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:43.331527 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:43.331527 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:43.331527 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:43.331527 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:43.331527 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:43.331527 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:43.331527 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 15:15:43.331527 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 15:15:43.331527 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 15:15:43.331527 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 15:15:43.332710 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:43.332710 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:43.332710 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 15:15:43.332710 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:15:43.332710 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 15:15:43.332710 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:15:43.332710 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:43.332710 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:43.332710 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:43.332710 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:43.332710 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:43.332710 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:43.332710 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:43.332710 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:43.332710 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:43.332710 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:43.332710 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:43.332710 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:43.333661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:43.333661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:43.333661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 15:15:43.333661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 15:15:43.333661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 15:15:43.333661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 15:15:43.333661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 15:15:43.333661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 15:15:43.333661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 07 15:15:43.333661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:43.333661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 307/628] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Jul 7 15:15:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:43.352634 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdbaee4-73d9-46f1-b314-0b94929a89a6 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:43.451819 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:43.459474 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 322/629] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:43.490335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c668d-21ab-4bc6-b4eb-4373684609d7 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:43.509152 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdbaee4-73d9-46f1-b314-0b94929a89a6 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:43.551996 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fdb173-17ac-4d47-a180-b7a857f50770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:43.552665 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fdb173-17ac-4d47-a180-b7a857f50770 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:43.552947 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fdb173-17ac-4d47-a180-b7a857f50770 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:43.553220 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fdb173-17ac-4d47-a180-b7a857f50770 None admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:43.554065 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fdb173-17ac-4d47-a180-b7a857f50770 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:43.554374 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fdb173-17ac-4d47-a180-b7a857f50770 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:43.556008 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fdb173-17ac-4d47-a180-b7a857f50770 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:43.588884 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 308/630] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Jul 7 15:15:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:15:43.594386 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fdb173-17ac-4d47-a180-b7a857f50770 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:43.662211 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c668d-21ab-4bc6-b4eb-4373684609d7 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:43.670200 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:43.712116 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c65c967-4c63-4e5f-9007-fa01366c0884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:43.712835 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65c967-4c63-4e5f-9007-fa01366c0884 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:43.713116 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65c967-4c63-4e5f-9007-fa01366c0884 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:43.713390 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65c967-4c63-4e5f-9007-fa01366c0884 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:43.725169 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c65c967-4c63-4e5f-9007-fa01366c0884 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:43.732978 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-4c65c967-4c63-4e5f-9007-fa01366c0884 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:15:43.732978 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:43.732978 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:43.732978 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:43.732978 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:43.732978 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:43.732978 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:43.732978 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:43.732978 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:43.732978 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:43.732978 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:43.732978 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:43.732978 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:43.732978 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:43.732978 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:43.732978 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:43.734489 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:43.734489 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:43.734489 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:43.734489 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:43.734489 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:43.734489 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:43.734489 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:43.734489 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:43.734489 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:43.734489 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:43.734489 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:43.734489 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:43.734489 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:43.734489 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:43.734489 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:43.734489 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:43.735342 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:43.735342 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:43.735342 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:43.735342 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:43.735342 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:43.735342 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:43.735342 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:43.735342 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:43.735342 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:43.735342 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:43.735342 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:15:43.735342 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:43.735342 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 323/631] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Jul 7 15:15:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:15:43.740595 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c65c967-4c63-4e5f-9007-fa01366c0884 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:43.804340 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:43.848470 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c65c967-4c63-4e5f-9007-fa01366c0884 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:43.857837 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fdb173-17ac-4d47-a180-b7a857f50770 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:43.890664 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c3d4c1-7a2f-4306-9981-c6703069a0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:43.891383 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c3d4c1-7a2f-4306-9981-c6703069a0ed None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:43.891696 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c3d4c1-7a2f-4306-9981-c6703069a0ed None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:43.891971 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c3d4c1-7a2f-4306-9981-c6703069a0ed None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:43.892724 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c3d4c1-7a2f-4306-9981-c6703069a0ed None admin] RBAC: Adding query filter params (name=service) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:43.893029 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c3d4c1-7a2f-4306-9981-c6703069a0ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:43.894416 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c3d4c1-7a2f-4306-9981-c6703069a0ed None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:43.903554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 324/632] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Jul 7 15:15:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:15:43.909487 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c3d4c1-7a2f-4306-9981-c6703069a0ed None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:43.968912 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f3615-28cf-4e52-b926-8a3a850b5025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:43.969528 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f3615-28cf-4e52-b926-8a3a850b5025 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:43.969810 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f3615-28cf-4e52-b926-8a3a850b5025 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:43.970081 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f3615-28cf-4e52-b926-8a3a850b5025 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:44.026424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c3d4c1-7a2f-4306-9981-c6703069a0ed None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:44.069933 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:44.073665 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d6674-9451-4c04-a035-312ce0221500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:44.074506 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d6674-9451-4c04-a035-312ce0221500 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:44.074790 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d6674-9451-4c04-a035-312ce0221500 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:44.075065 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d6674-9451-4c04-a035-312ce0221500 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:44.086386 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1d6674-9451-4c04-a035-312ce0221500 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:44.087895 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1d6674-9451-4c04-a035-312ce0221500 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:44.108117 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-cd1d6674-9451-4c04-a035-312ce0221500 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:44.108117 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:44.108117 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:44.108117 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:44.108117 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:44.108117 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:44.108117 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:44.108117 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:44.108117 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:44.108117 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:44.108117 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:44.108117 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:44.108117 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:44.108117 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:44.108117 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:44.109682 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:44.109682 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:44.109682 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:44.109682 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:44.109682 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:44.109682 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:44.109682 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:44.109682 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:44.109682 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:44.109682 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:44.109682 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:44.109682 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:44.109682 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:44.109682 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:44.109682 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:44.109682 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:44.109682 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:44.110574 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:44.110574 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:44.110574 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:44.110574 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:44.110574 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 325/633] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:44.112226 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03bf31b-dd4e-445a-9220-bd689294f403 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:44.115940 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03bf31b-dd4e-445a-9220-bd689294f403 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:44.116255 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03bf31b-dd4e-445a-9220-bd689294f403 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:44.117843 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 326/634] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:44.119085 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03bf31b-dd4e-445a-9220-bd689294f403 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:44.130324 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172f3615-28cf-4e52-b926-8a3a850b5025 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:44.132569 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172f3615-28cf-4e52-b926-8a3a850b5025 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:44.181710 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:44.225483 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03bf31b-dd4e-445a-9220-bd689294f403 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:44.248150 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 309/635] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:15:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 654 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 15:15:44.249486 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13af44-6e5d-44a8-bf52-918dfff2798b None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:44.249770 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13af44-6e5d-44a8-bf52-918dfff2798b None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:44.250042 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13af44-6e5d-44a8-bf52-918dfff2798b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:44.264116 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57055d67-e835-435b-95c8-3c03aadc4a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:44.265293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57055d67-e835-435b-95c8-3c03aadc4a96 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:44.265596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57055d67-e835-435b-95c8-3c03aadc4a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:44.265872 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57055d67-e835-435b-95c8-3c03aadc4a96 None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:44.266598 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57055d67-e835-435b-95c8-3c03aadc4a96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:44.266906 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57055d67-e835-435b-95c8-3c03aadc4a96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:44.267012 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-2a13af44-6e5d-44a8-bf52-918dfff2798b None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:44.268152 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-2a13af44-6e5d-44a8-bf52-918dfff2798b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:44.268330 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57055d67-e835-435b-95c8-3c03aadc4a96 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:44.286041 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 327/636] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:44.286041 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b3d18-69fd-4b5b-b387-22e605a37e8c None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:44.286041 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b3d18-69fd-4b5b-b387-22e605a37e8c None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:44.286041 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b3d18-69fd-4b5b-b387-22e605a37e8c None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:44.286041 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 328/637] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:44.286041 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b3d18-69fd-4b5b-b387-22e605a37e8c None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:44.330938 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:44.359759 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 310/638] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:44.359759 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13af44-6e5d-44a8-bf52-918dfff2798b None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:44.410075 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b3d18-69fd-4b5b-b387-22e605a37e8c None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:44.442628 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:44.447591 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f18eba-300a-4d4a-b59f-a9d17e17bfa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:44.448392 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f18eba-300a-4d4a-b59f-a9d17e17bfa0 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:44.448669 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f18eba-300a-4d4a-b59f-a9d17e17bfa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:44.448938 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f18eba-300a-4d4a-b59f-a9d17e17bfa0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:44.460030 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f18eba-300a-4d4a-b59f-a9d17e17bfa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:44.468791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f18eba-300a-4d4a-b59f-a9d17e17bfa0 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:44.477213 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13af44-6e5d-44a8-bf52-918dfff2798b None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:44.480317 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-81f18eba-300a-4d4a-b59f-a9d17e17bfa0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:44.480317 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:44.480317 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:44.480317 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:44.480317 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:44.480317 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:44.480317 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:44.480317 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:44.480317 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:44.480317 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:44.480317 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:44.480317 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:44.480317 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:44.480317 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:44.480317 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:44.481210 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:44.481210 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:44.481210 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:44.481210 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:44.481210 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:44.481210 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:44.481210 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:44.481210 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:44.481210 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:44.481210 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:44.481210 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:44.481210 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:44.481210 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:44.481210 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:44.481210 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:44.481210 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:44.481210 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:44.482107 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:44.482107 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:44.482107 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:44.482107 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:44.486196 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 329/639] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:44.487330 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8643a57-8718-4813-8691-05a392140b26 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:44.487668 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8643a57-8718-4813-8691-05a392140b26 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:44.487959 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8643a57-8718-4813-8691-05a392140b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:44.504439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-f8643a57-8718-4813-8691-05a392140b26 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:44.505627 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-f8643a57-8718-4813-8691-05a392140b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:44.512038 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab144db-d424-47ae-b0c1-2f5a8c2e822e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:44.513142 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab144db-d424-47ae-b0c1-2f5a8c2e822e None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:44.513490 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab144db-d424-47ae-b0c1-2f5a8c2e822e None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:44.513763 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab144db-d424-47ae-b0c1-2f5a8c2e822e None admin] PATH_INFO: `/v3/users/glance` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:44.521955 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab144db-d424-47ae-b0c1-2f5a8c2e822e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:44.523616 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab144db-d424-47ae-b0c1-2f5a8c2e822e None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:44.539057 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-8ab144db-d424-47ae-b0c1-2f5a8c2e822e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 07 15:15:44.539057 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:44.539057 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:44.539057 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:44.539057 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:44.539057 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:44.539057 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:44.539057 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:44.539057 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:44.539057 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:44.539057 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:44.539057 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:44.539057 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 15:15:44.539057 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 15:15:44.539057 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 15:15:44.539057 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 15:15:44.540051 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:44.540051 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:44.540051 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 15:15:44.540051 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:15:44.540051 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 15:15:44.540051 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:15:44.540051 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:44.540051 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:44.540051 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:44.540051 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:44.540051 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:44.540051 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:44.540051 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:44.540051 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:44.540051 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:44.540051 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:44.540051 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:44.540051 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:44.540990 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:44.540990 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:44.540990 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 15:15:44.540990 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 15:15:44.540990 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 15:15:44.540990 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 15:15:44.540990 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 15:15:44.540990 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 15:15:44.540990 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 07 15:15:44.540990 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:44.540990 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 311/640] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:15:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:44.540990 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab144db-d424-47ae-b0c1-2f5a8c2e822e None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:44.581844 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 330/641] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:44.582761 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8643a57-8718-4813-8691-05a392140b26 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:44.606704 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:44.652342 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab144db-d424-47ae-b0c1-2f5a8c2e822e None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:44.692415 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:44.701164 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8643a57-8718-4813-8691-05a392140b26 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:44.710247 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb1a15d-b57d-4497-8bcf-9d7cc04063c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:44.711039 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1a15d-b57d-4497-8bcf-9d7cc04063c8 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:44.711372 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1a15d-b57d-4497-8bcf-9d7cc04063c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:44.711674 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1a15d-b57d-4497-8bcf-9d7cc04063c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:44.723168 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb1a15d-b57d-4497-8bcf-9d7cc04063c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:44.731142 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-adb1a15d-b57d-4497-8bcf-9d7cc04063c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:15:44.731142 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:44.731142 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:44.731142 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:44.731142 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:44.731142 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:44.731142 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:44.731142 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:44.731142 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:44.731142 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:44.731142 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:44.731142 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:44.731142 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:44.731142 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:44.731142 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:44.731142 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:44.732393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:44.732393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:44.732393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:44.732393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:44.732393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:44.732393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:44.732393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:44.732393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:44.732393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:44.732393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:44.732393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:44.732393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:44.732393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:44.732393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:44.732393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:44.732393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:44.733297 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:44.733297 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:44.733297 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:44.733297 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:44.733297 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:44.733297 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:44.733297 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:44.733297 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:44.733297 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:44.733297 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:44.733297 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:15:44.733297 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:44.736074 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 312/642] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Jul 7 15:15:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:44.737031 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb1a15d-b57d-4497-8bcf-9d7cc04063c8 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:44.744737 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1aac96-7577-4f49-8b6d-5b07119bc085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:44.745407 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1aac96-7577-4f49-8b6d-5b07119bc085 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:44.745691 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1aac96-7577-4f49-8b6d-5b07119bc085 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:44.745966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1aac96-7577-4f49-8b6d-5b07119bc085 None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:44.746701 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1aac96-7577-4f49-8b6d-5b07119bc085 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:44.747007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1aac96-7577-4f49-8b6d-5b07119bc085 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:44.748404 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1aac96-7577-4f49-8b6d-5b07119bc085 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:44.760566 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 331/643] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:44.761206 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1aac96-7577-4f49-8b6d-5b07119bc085 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:44.842587 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:44.847509 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb1a15d-b57d-4497-8bcf-9d7cc04063c8 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:44.866613 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1aac96-7577-4f49-8b6d-5b07119bc085 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:44.884613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362bf42-f7f5-42fb-b741-e25c071ae617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:44.888853 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362bf42-f7f5-42fb-b741-e25c071ae617 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:44.888853 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362bf42-f7f5-42fb-b741-e25c071ae617 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:44.888853 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362bf42-f7f5-42fb-b741-e25c071ae617 None admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:44.888853 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362bf42-f7f5-42fb-b741-e25c071ae617 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:44.888853 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362bf42-f7f5-42fb-b741-e25c071ae617 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:44.888853 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362bf42-f7f5-42fb-b741-e25c071ae617 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:44.902956 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd79a80b-becc-412e-96e0-6325675e6eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:44.903775 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79a80b-becc-412e-96e0-6325675e6eae None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:44.904055 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79a80b-becc-412e-96e0-6325675e6eae None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:44.904332 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79a80b-becc-412e-96e0-6325675e6eae None admin] PATH_INFO: `/v3/roles/service` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:44.908330 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 313/644] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Jul 7 15:15:44 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:44.909646 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362bf42-f7f5-42fb-b741-e25c071ae617 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:44.917369 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd79a80b-becc-412e-96e0-6325675e6eae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:44.925144 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-cd79a80b-becc-412e-96e0-6325675e6eae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:15:44.925144 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:44.925144 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:44.925144 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:44.925144 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:44.925144 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:44.925144 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:44.925144 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:44.925144 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:44.925144 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:44.925144 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:44.925144 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:44.925144 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:44.925144 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:44.925144 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:44.925144 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:44.926187 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:44.926187 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:44.926187 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:44.926187 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:44.926187 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:44.926187 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:44.926187 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:44.926187 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:44.926187 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:44.926187 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:44.926187 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:44.926187 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:44.926187 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:44.926187 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:44.926187 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:44.926187 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:44.932299 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:44.932299 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:44.932299 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:44.932299 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:44.932299 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:44.932299 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:44.932299 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:44.932299 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:44.932299 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:44.932299 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:44.932299 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:15:44.932299 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:44.932299 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 332/645] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Jul 7 15:15:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:44.932299 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd79a80b-becc-412e-96e0-6325675e6eae None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:44.970088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:45.032387 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362bf42-f7f5-42fb-b741-e25c071ae617 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:45.043964 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd79a80b-becc-412e-96e0-6325675e6eae None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:45.073814 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e2c685-9225-445b-b620-0ae7b1848a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:45.074726 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2c685-9225-445b-b620-0ae7b1848a82 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:45.075009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2c685-9225-445b-b620-0ae7b1848a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:45.075273 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2c685-9225-445b-b620-0ae7b1848a82 None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:45.076060 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e2c685-9225-445b-b620-0ae7b1848a82 None admin] RBAC: Adding query filter params (name=service) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:45.076460 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e2c685-9225-445b-b620-0ae7b1848a82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:45.077772 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e2c685-9225-445b-b620-0ae7b1848a82 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:45.080910 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5923f97b-28c0-4a4e-9e3d-5a7377649a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:45.081723 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923f97b-28c0-4a4e-9e3d-5a7377649a18 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:45.082005 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923f97b-28c0-4a4e-9e3d-5a7377649a18 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:45.082277 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923f97b-28c0-4a4e-9e3d-5a7377649a18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:45.085763 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:45.090327 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5923f97b-28c0-4a4e-9e3d-5a7377649a18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:45.090755 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 314/646] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Jul 7 15:15:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:45.091542 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e2c685-9225-445b-b620-0ae7b1848a82 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:45.091739 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5923f97b-28c0-4a4e-9e3d-5a7377649a18 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:45.107103 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-5923f97b-28c0-4a4e-9e3d-5a7377649a18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:15:45.107103 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:45.107103 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:45.107103 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:45.107103 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:45.107103 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:45.107103 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:45.107103 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:45.107103 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:45.107103 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:45.107103 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:45.107103 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:45.107103 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 15:15:45.107103 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 15:15:45.107103 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 15:15:45.108090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 15:15:45.108090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:45.108090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:45.108090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:45.108090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:45.108090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:45.108090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:45.108090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:45.108090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:45.108090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:45.108090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:45.108090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:45.108090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:45.108090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:45.108090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:45.108090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:45.108090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:45.109511 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 15:15:45.109511 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 15:15:45.109511 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 15:15:45.109511 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 15:15:45.109511 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 15:15:45.109511 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 15:15:45.109511 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:15:45.109511 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:45.109511 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 333/647] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:15:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:45.109994 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5923f97b-28c0-4a4e-9e3d-5a7377649a18 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:45.206779 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e2c685-9225-445b-b620-0ae7b1848a82 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:45.215360 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5923f97b-28c0-4a4e-9e3d-5a7377649a18 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:45.217039 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:45.244359 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abeb2c9-d304-42f8-bb26-0bd6f9a5a95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:45.245119 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abeb2c9-d304-42f8-bb26-0bd6f9a5a95d None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:45.245427 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abeb2c9-d304-42f8-bb26-0bd6f9a5a95d None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:45.245838 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abeb2c9-d304-42f8-bb26-0bd6f9a5a95d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:45.251083 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5836527a-126b-4676-88f9-b5598c981541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:45.251785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5836527a-126b-4676-88f9-b5598c981541 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:45.252065 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5836527a-126b-4676-88f9-b5598c981541 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:45.252334 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5836527a-126b-4676-88f9-b5598c981541 None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:45.253077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5836527a-126b-4676-88f9-b5598c981541 None admin] RBAC: Adding query filter params (name=service) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:45.253383 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5836527a-126b-4676-88f9-b5598c981541 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:45.254848 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5836527a-126b-4676-88f9-b5598c981541 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:45.262784 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 334/648] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Jul 7 15:15:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:45.263556 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5836527a-126b-4676-88f9-b5598c981541 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:45.320805 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:45.322680 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abeb2c9-d304-42f8-bb26-0bd6f9a5a95d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:45.324161 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abeb2c9-d304-42f8-bb26-0bd6f9a5a95d None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:45.365011 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5836527a-126b-4676-88f9-b5598c981541 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:45.398833 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06755f89-c35d-43e9-8f0f-29cd17ac6a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:45.399649 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06755f89-c35d-43e9-8f0f-29cd17ac6a46 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:45.399937 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06755f89-c35d-43e9-8f0f-29cd17ac6a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:45.400209 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06755f89-c35d-43e9-8f0f-29cd17ac6a46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:45.411977 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06755f89-c35d-43e9-8f0f-29cd17ac6a46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:45.413421 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06755f89-c35d-43e9-8f0f-29cd17ac6a46 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:45.433474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 315/649] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:15:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 15:15:45.433474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abeb2c9-d304-42f8-bb26-0bd6f9a5a95d None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:45.433738 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-06755f89-c35d-43e9-8f0f-29cd17ac6a46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:45.433738 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:45.433738 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:45.433738 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:45.433738 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:45.433738 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:45.433738 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:45.433738 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:45.433738 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:45.433738 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:45.433738 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:45.433738 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:45.433738 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:45.433738 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:45.433738 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:45.434546 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:45.434546 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:45.434546 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:45.434546 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:45.434546 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:45.434546 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:45.434546 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:45.434546 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:45.434546 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:45.434546 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:45.434546 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:45.434546 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:45.434546 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:45.434546 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:45.434546 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:45.434546 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:45.434546 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:45.435447 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:45.435447 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:45.435447 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:45.435447 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:45.435447 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 335/650] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:45.435447 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06755f89-c35d-43e9-8f0f-29cd17ac6a46 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:45.445800 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:45.540422 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abeb2c9-d304-42f8-bb26-0bd6f9a5a95d None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:45.541042 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06755f89-c35d-43e9-8f0f-29cd17ac6a46 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:45.552222 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:45.574879 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5970b142-969e-4bd6-ac60-ac814ba04457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:45.575618 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970b142-969e-4bd6-ac60-ac814ba04457 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:45.575902 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970b142-969e-4bd6-ac60-ac814ba04457 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:45.576173 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970b142-969e-4bd6-ac60-ac814ba04457 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:45.576359 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f579b-ad27-4c08-96a3-82c9fa6c285b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:45.577033 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f579b-ad27-4c08-96a3-82c9fa6c285b None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:45.577351 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f579b-ad27-4c08-96a3-82c9fa6c285b None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:45.577626 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f579b-ad27-4c08-96a3-82c9fa6c285b None admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:45.578473 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f579b-ad27-4c08-96a3-82c9fa6c285b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:45.578787 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f579b-ad27-4c08-96a3-82c9fa6c285b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:45.580250 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f579b-ad27-4c08-96a3-82c9fa6c285b None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:45.586948 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5970b142-969e-4bd6-ac60-ac814ba04457 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:45.588421 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5970b142-969e-4bd6-ac60-ac814ba04457 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:45.592115 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 316/651] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Jul 7 15:15:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:45.593436 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f579b-ad27-4c08-96a3-82c9fa6c285b None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:45.606164 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-5970b142-969e-4bd6-ac60-ac814ba04457 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:45.606164 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:45.606164 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:45.606164 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:45.606164 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:45.606164 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:45.606164 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:45.606164 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:45.606164 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:45.606164 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:45.606164 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:45.606164 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:45.606164 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:45.606164 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:45.606164 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:45.607079 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:45.607079 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:45.607079 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:45.607079 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:45.607079 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:45.607079 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:45.607079 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:45.607079 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:45.607079 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:45.607079 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:45.607079 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:45.607079 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:45.607079 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:45.607079 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:45.607079 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:45.607079 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:45.607079 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:45.608023 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:45.608023 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:45.608023 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:45.608023 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:45.608023 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 336/652] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:45.609099 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5970b142-969e-4bd6-ac60-ac814ba04457 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:45.662979 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:45.695754 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f579b-ad27-4c08-96a3-82c9fa6c285b None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:45.707986 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5970b142-969e-4bd6-ac60-ac814ba04457 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:45.729737 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d167c6f1-2299-4f33-8783-0053f5869f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:45.730518 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167c6f1-2299-4f33-8783-0053f5869f87 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:45.730803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167c6f1-2299-4f33-8783-0053f5869f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:45.731080 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167c6f1-2299-4f33-8783-0053f5869f87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:45.741342 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c6df38-3ac7-4482-9862-59c1e9755fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:45.741533 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d167c6f1-2299-4f33-8783-0053f5869f87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:45.741986 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6df38-3ac7-4482-9862-59c1e9755fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:45.742269 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6df38-3ac7-4482-9862-59c1e9755fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:45.742542 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6df38-3ac7-4482-9862-59c1e9755fa8 None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:45.742959 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d167c6f1-2299-4f33-8783-0053f5869f87 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:45.743255 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c6df38-3ac7-4482-9862-59c1e9755fa8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:45.743582 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c6df38-3ac7-4482-9862-59c1e9755fa8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:45.744946 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c6df38-3ac7-4482-9862-59c1e9755fa8 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:45.756375 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 337/653] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:45.757576 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fa4f3-8955-47f0-ab65-eda3ce2c07f0 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:45.757861 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fa4f3-8955-47f0-ab65-eda3ce2c07f0 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:45.758136 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fa4f3-8955-47f0-ab65-eda3ce2c07f0 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:45.759554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 338/654] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:45.759689 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-d167c6f1-2299-4f33-8783-0053f5869f87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:45.759689 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:45.759689 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:45.759689 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:45.759689 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:45.759689 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:45.759689 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:45.759689 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:45.759689 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:45.759689 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:45.759689 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:45.759689 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:45.759689 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:45.759689 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:45.759689 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:45.760794 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:45.760794 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:45.760794 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:45.760794 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:45.760794 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:45.760794 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:45.760794 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:45.760794 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:45.760794 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:45.760794 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:45.760794 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:45.760794 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:45.760794 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:45.760794 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:45.760794 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:45.760794 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:45.760794 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:45.761989 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:45.761989 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:45.761989 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:45.761989 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:45.761989 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 317/655] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:45.761989 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d167c6f1-2299-4f33-8783-0053f5869f87 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:45.764004 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29fa4f3-8955-47f0-ab65-eda3ce2c07f0 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:45.788268 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:45.860762 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d167c6f1-2299-4f33-8783-0053f5869f87 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:45.865532 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29fa4f3-8955-47f0-ab65-eda3ce2c07f0 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:45.879891 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:45.894293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0081efce-0762-42d1-8392-bde8fac25079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:45.894953 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081efce-0762-42d1-8392-bde8fac25079 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:45.895236 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081efce-0762-42d1-8392-bde8fac25079 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:45.895512 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081efce-0762-42d1-8392-bde8fac25079 None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:45.896268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0081efce-0762-42d1-8392-bde8fac25079 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:45.896577 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0081efce-0762-42d1-8392-bde8fac25079 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:45.898000 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0081efce-0762-42d1-8392-bde8fac25079 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:45.900920 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a603f4-46e7-49ad-ab86-6fbda91e24bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:45.901573 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a603f4-46e7-49ad-ab86-6fbda91e24bb None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:45.901859 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a603f4-46e7-49ad-ab86-6fbda91e24bb None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:45.902140 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a603f4-46e7-49ad-ab86-6fbda91e24bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:45.902975 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a603f4-46e7-49ad-ab86-6fbda91e24bb None admin] RBAC: Adding query filter params (role.id=3d4f4e0a472e4ff9ac68d90c8612e956, scope.project.id=c5260c7fd3614e9e93b167c86a94260e, user.id=de1632555cb345f2b9213da1925c02f4) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:45.903289 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a603f4-46e7-49ad-ab86-6fbda91e24bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:45.904983 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a603f4-46e7-49ad-ab86-6fbda91e24bb None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:45.910783 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 318/656] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:45.922842 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 339/657] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:15:45 2022] GET /identity/v3/role_assignments?user.id=de1632555cb345f2b9213da1925c02f4&scope.project.id=c5260c7fd3614e9e93b167c86a94260e&role.id=3d4f4e0a472e4ff9ac68d90c8612e956 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:45.923104 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb9928-d382-4323-8c48-0e84eae4590e None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:45.923104 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb9928-d382-4323-8c48-0e84eae4590e None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:45.923104 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb9928-d382-4323-8c48-0e84eae4590e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:45.923306 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a603f4-46e7-49ad-ab86-6fbda91e24bb None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:45.932512 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-09fb9928-d382-4323-8c48-0e84eae4590e None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:45.933843 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-09fb9928-d382-4323-8c48-0e84eae4590e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:46.017901 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:46.033386 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 319/658] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:46.033386 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb9928-d382-4323-8c48-0e84eae4590e None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:46.049200 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a603f4-46e7-49ad-ab86-6fbda91e24bb None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:46.094234 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01662ce4-af63-4c24-b499-a28e1569214b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:46.094896 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01662ce4-af63-4c24-b499-a28e1569214b None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:46.095178 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01662ce4-af63-4c24-b499-a28e1569214b None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:46.095446 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01662ce4-af63-4c24-b499-a28e1569214b None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:46.096207 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01662ce4-af63-4c24-b499-a28e1569214b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:46.096505 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01662ce4-af63-4c24-b499-a28e1569214b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:46.098257 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01662ce4-af63-4c24-b499-a28e1569214b None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:46.111909 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 340/659] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:46.113011 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01662ce4-af63-4c24-b499-a28e1569214b None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:46.145488 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb9928-d382-4323-8c48-0e84eae4590e None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:46.164432 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:46.181394 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3133a9-7a2a-494a-87d5-3489f6b300db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:46.182181 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3133a9-7a2a-494a-87d5-3489f6b300db None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:46.182462 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3133a9-7a2a-494a-87d5-3489f6b300db None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:46.182734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3133a9-7a2a-494a-87d5-3489f6b300db None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:46.190612 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3133a9-7a2a-494a-87d5-3489f6b300db None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:46.192079 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3133a9-7a2a-494a-87d5-3489f6b300db None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:46.207159 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-3b3133a9-7a2a-494a-87d5-3489f6b300db None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 07 15:15:46.207159 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:46.207159 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:46.207159 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:46.207159 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:46.207159 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:46.207159 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:46.207159 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:46.207159 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:46.207159 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:46.207159 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:46.207159 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:46.207159 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 15:15:46.207159 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 15:15:46.207159 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 15:15:46.207159 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 15:15:46.208353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:46.208353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:46.208353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 15:15:46.208353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:15:46.208353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 15:15:46.208353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:15:46.208353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:46.208353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:46.208353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:46.208353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:46.208353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:46.208353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:46.208353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:46.208353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:46.208353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:46.208353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:46.208353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:46.208353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:46.209304 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:46.209304 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:46.209304 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 15:15:46.209304 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 15:15:46.209304 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 15:15:46.209304 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 15:15:46.209304 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 15:15:46.209304 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 15:15:46.209304 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 07 15:15:46.209304 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:46.209304 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 320/660] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:15:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:46.209304 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3133a9-7a2a-494a-87d5-3489f6b300db None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:46.234302 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01662ce4-af63-4c24-b499-a28e1569214b None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:46.270914 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf8702-634f-432a-8444-35ce841c2ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:46.271776 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf8702-634f-432a-8444-35ce841c2ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:46.272025 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf8702-634f-432a-8444-35ce841c2ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:46.272299 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf8702-634f-432a-8444-35ce841c2ab9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:46.279746 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cf8702-634f-432a-8444-35ce841c2ab9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:46.281218 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cf8702-634f-432a-8444-35ce841c2ab9 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:46.296016 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-a4cf8702-634f-432a-8444-35ce841c2ab9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 07 15:15:46.296016 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:46.296016 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:46.296016 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:46.296016 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:46.296016 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:46.296016 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:46.296016 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:46.296016 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:46.296016 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:46.296016 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:46.296016 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:46.296016 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 15:15:46.296016 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 15:15:46.296016 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 15:15:46.296016 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 15:15:46.297048 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:46.297048 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:46.297048 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 15:15:46.297048 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:15:46.297048 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 15:15:46.297048 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:15:46.297048 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:46.297048 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:46.297048 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:46.297048 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:46.297048 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:46.297048 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:46.297048 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:46.297048 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:46.297048 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:46.297048 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:46.297048 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:46.297048 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:46.297999 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:46.297999 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:46.297999 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 15:15:46.297999 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 15:15:46.297999 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 15:15:46.297999 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 15:15:46.297999 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 15:15:46.297999 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 15:15:46.297999 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 07 15:15:46.297999 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:46.297999 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 341/661] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Jul 7 15:15:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:46.297999 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf8702-634f-432a-8444-35ce841c2ab9 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:46.307065 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:46.322379 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3133a9-7a2a-494a-87d5-3489f6b300db None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:46.359586 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023d2f0-8918-4eac-bb3e-b8584dd3064f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:46.360396 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023d2f0-8918-4eac-bb3e-b8584dd3064f None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:46.360676 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023d2f0-8918-4eac-bb3e-b8584dd3064f None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:46.360950 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023d2f0-8918-4eac-bb3e-b8584dd3064f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:46.371394 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023d2f0-8918-4eac-bb3e-b8584dd3064f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:46.379679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-d023d2f0-8918-4eac-bb3e-b8584dd3064f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:15:46.379679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:46.379679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:46.379679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:46.379679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:46.379679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:46.379679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:46.379679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:46.379679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:46.379679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:46.379679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:46.379679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:46.379679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:46.379679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:46.379679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:46.379679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:46.380630 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:46.380630 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:46.380630 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:46.380630 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:46.380630 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:46.380630 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:46.380630 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:46.380630 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:46.380630 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:46.380630 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:46.380630 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:46.380630 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:46.380630 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:46.380630 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:46.380630 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:46.380630 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:46.381475 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:46.381475 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:46.381475 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:46.381475 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:46.381475 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:46.381475 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:46.381475 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:46.381475 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:46.381475 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:46.381475 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:46.381475 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:15:46.381475 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:46.381475 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 321/662] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Jul 7 15:15:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:46.382254 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023d2f0-8918-4eac-bb3e-b8584dd3064f None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:46.412820 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf8702-634f-432a-8444-35ce841c2ab9 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:46.414000 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:46.448917 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a338e1-713f-4550-821f-24a7363ea6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:46.449704 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a338e1-713f-4550-821f-24a7363ea6cd None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:46.449981 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a338e1-713f-4550-821f-24a7363ea6cd None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:46.450285 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a338e1-713f-4550-821f-24a7363ea6cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:46.457878 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a338e1-713f-4550-821f-24a7363ea6cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:46.459306 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a338e1-713f-4550-821f-24a7363ea6cd None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:46.473385 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-f3a338e1-713f-4550-821f-24a7363ea6cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:15:46.473385 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:46.473385 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:46.473385 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:46.473385 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:46.473385 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:46.473385 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:46.473385 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:46.473385 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:46.473385 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:46.473385 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:46.473385 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:46.473385 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 15:15:46.473385 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 15:15:46.473385 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 15:15:46.474367 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 15:15:46.474367 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:46.474367 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:46.474367 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:46.474367 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:46.474367 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:46.474367 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:46.474367 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:46.474367 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:46.474367 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:46.474367 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:46.474367 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:46.474367 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:46.474367 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:46.474367 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:46.474367 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:46.474367 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:46.475273 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 15:15:46.475273 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 15:15:46.475273 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 15:15:46.475273 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 15:15:46.475273 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 15:15:46.475273 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 15:15:46.475273 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:15:46.475273 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:46.475273 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 342/663] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:15:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:46.475848 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a338e1-713f-4550-821f-24a7363ea6cd None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:46.504644 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023d2f0-8918-4eac-bb3e-b8584dd3064f None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:46.540056 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c92a2-1ee0-414d-baae-f8a865d679a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:46.540716 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c92a2-1ee0-414d-baae-f8a865d679a2 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:46.540993 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c92a2-1ee0-414d-baae-f8a865d679a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:46.541275 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c92a2-1ee0-414d-baae-f8a865d679a2 None admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:46.542108 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9c92a2-1ee0-414d-baae-f8a865d679a2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:46.542416 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9c92a2-1ee0-414d-baae-f8a865d679a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:46.543872 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9c92a2-1ee0-414d-baae-f8a865d679a2 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:46.544907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:46.566475 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 322/664] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Jul 7 15:15:46 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:46.566475 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c92a2-1ee0-414d-baae-f8a865d679a2 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:46.585566 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a338e1-713f-4550-821f-24a7363ea6cd None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:46.621325 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cba433-ef02-4acb-b29b-ab828f2b47a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:46.622046 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cba433-ef02-4acb-b29b-ab828f2b47a0 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:46.622272 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cba433-ef02-4acb-b29b-ab828f2b47a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:46.622548 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cba433-ef02-4acb-b29b-ab828f2b47a0 None admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:46.623376 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cba433-ef02-4acb-b29b-ab828f2b47a0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:46.623716 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cba433-ef02-4acb-b29b-ab828f2b47a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:46.625692 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cba433-ef02-4acb-b29b-ab828f2b47a0 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:46.647366 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 343/665] 10.222.0.25 () {56 vars in 1139 bytes} [Thu Jul 7 15:15:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:46.647366 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cba433-ef02-4acb-b29b-ab828f2b47a0 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:46.659148 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:46.672080 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c92a2-1ee0-414d-baae-f8a865d679a2 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:46.710111 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9417aac-aaee-4431-8c92-54e7c89b75b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:46.711110 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9417aac-aaee-4431-8c92-54e7c89b75b9 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:46.711656 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9417aac-aaee-4431-8c92-54e7c89b75b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:46.712152 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9417aac-aaee-4431-8c92-54e7c89b75b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:46.713101 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9417aac-aaee-4431-8c92-54e7c89b75b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:46.713623 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9417aac-aaee-4431-8c92-54e7c89b75b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:46.715196 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9417aac-aaee-4431-8c92-54e7c89b75b9 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:46.727611 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 323/666] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Jul 7 15:15:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:46.727611 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9417aac-aaee-4431-8c92-54e7c89b75b9 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:46.754795 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cba433-ef02-4acb-b29b-ab828f2b47a0 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:46.779435 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:46.789939 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1848f107-c1a7-4ab4-a2a5-460261e78f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:46.790590 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848f107-c1a7-4ab4-a2a5-460261e78f13 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:46.804988 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848f107-c1a7-4ab4-a2a5-460261e78f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:46.804988 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848f107-c1a7-4ab4-a2a5-460261e78f13 None admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:46.804988 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1848f107-c1a7-4ab4-a2a5-460261e78f13 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:46.804988 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1848f107-c1a7-4ab4-a2a5-460261e78f13 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:46.804988 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1848f107-c1a7-4ab4-a2a5-460261e78f13 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:46.810058 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 344/667] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Jul 7 15:15:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:46.810945 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1848f107-c1a7-4ab4-a2a5-460261e78f13 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:46.831350 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9417aac-aaee-4431-8c92-54e7c89b75b9 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:46.868597 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a567d1cf-fc8a-42d9-933a-df867b5c855b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:46.869248 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567d1cf-fc8a-42d9-933a-df867b5c855b None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:46.869535 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567d1cf-fc8a-42d9-933a-df867b5c855b None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:46.869794 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567d1cf-fc8a-42d9-933a-df867b5c855b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:46.889485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:46.914009 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1848f107-c1a7-4ab4-a2a5-460261e78f13 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:46.942121 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a567d1cf-fc8a-42d9-933a-df867b5c855b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:46.944014 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a567d1cf-fc8a-42d9-933a-df867b5c855b None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:46.949109 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747a806e-bdfa-4575-8d73-991d32dfdf28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:46.949765 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a806e-bdfa-4575-8d73-991d32dfdf28 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:46.950047 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a806e-bdfa-4575-8d73-991d32dfdf28 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:46.950320 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a806e-bdfa-4575-8d73-991d32dfdf28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:47.011068 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:47.027067 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747a806e-bdfa-4575-8d73-991d32dfdf28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:47.028572 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747a806e-bdfa-4575-8d73-991d32dfdf28 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:47.054925 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 324/668] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:15:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 15:15:47.056143 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a567d1cf-fc8a-42d9-933a-df867b5c855b None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:47.113263 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:47.150709 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 345/669] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:15:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 15:15:47.150709 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747a806e-bdfa-4575-8d73-991d32dfdf28 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:47.169565 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a567d1cf-fc8a-42d9-933a-df867b5c855b None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:47.204426 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e22be9-3a61-4e94-ae20-cde4048ef341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:47.205199 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e22be9-3a61-4e94-ae20-cde4048ef341 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:47.205477 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e22be9-3a61-4e94-ae20-cde4048ef341 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:47.205750 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e22be9-3a61-4e94-ae20-cde4048ef341 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:47.216182 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e22be9-3a61-4e94-ae20-cde4048ef341 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:47.217605 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e22be9-3a61-4e94-ae20-cde4048ef341 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:47.235082 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:47.236236 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-c4e22be9-3a61-4e94-ae20-cde4048ef341 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:47.236236 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:47.236236 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:47.236236 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:47.236236 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:47.236236 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:47.236236 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:47.236236 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:47.236236 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:47.236236 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:47.236236 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:47.236236 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:47.236236 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:47.236236 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:47.236236 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:47.237283 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:47.237283 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:47.237283 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:47.237283 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:47.237283 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:47.237283 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:47.237283 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:47.237283 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:47.237283 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:47.237283 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:47.237283 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:47.237283 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:47.237283 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:47.237283 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:47.237283 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:47.237283 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:47.237283 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:47.238224 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:47.238224 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:47.238224 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:47.238224 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:47.238224 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 325/670] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:47.238837 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e22be9-3a61-4e94-ae20-cde4048ef341 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:47.253274 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747a806e-bdfa-4575-8d73-991d32dfdf28 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:47.289086 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fca79d4-53f3-4a8e-9a48-8e49b0ed91d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:47.289741 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca79d4-53f3-4a8e-9a48-8e49b0ed91d2 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:47.290018 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca79d4-53f3-4a8e-9a48-8e49b0ed91d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:47.290294 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca79d4-53f3-4a8e-9a48-8e49b0ed91d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:47.291113 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fca79d4-53f3-4a8e-9a48-8e49b0ed91d2 None admin] RBAC: Adding query filter params (role.id=3d4f4e0a472e4ff9ac68d90c8612e956, scope.project.id=c5260c7fd3614e9e93b167c86a94260e, user.id=3d0d29e718f446e2a9fd1b2575a86041) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:47.291423 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fca79d4-53f3-4a8e-9a48-8e49b0ed91d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:47.292864 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fca79d4-53f3-4a8e-9a48-8e49b0ed91d2 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:47.299412 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 346/671] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:15:47 2022] GET /identity/v3/role_assignments?user.id=3d0d29e718f446e2a9fd1b2575a86041&scope.project.id=c5260c7fd3614e9e93b167c86a94260e&role.id=3d4f4e0a472e4ff9ac68d90c8612e956 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:47.305242 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fca79d4-53f3-4a8e-9a48-8e49b0ed91d2 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:47.355653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e22be9-3a61-4e94-ae20-cde4048ef341 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:47.357677 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:47.396955 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767081e-485f-4204-b4da-ae1029aad211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:47.397755 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767081e-485f-4204-b4da-ae1029aad211 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:47.398033 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767081e-485f-4204-b4da-ae1029aad211 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:47.398304 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767081e-485f-4204-b4da-ae1029aad211 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:47.410410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e767081e-485f-4204-b4da-ae1029aad211 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:47.411816 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e767081e-485f-4204-b4da-ae1029aad211 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:47.437310 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-e767081e-485f-4204-b4da-ae1029aad211 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:47.437310 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:47.437310 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:47.437310 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:47.437310 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:47.437310 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:47.437310 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:47.437310 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:47.437310 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:47.437310 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:47.437310 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:47.437310 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:47.437310 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:47.437310 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:47.437310 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:47.438309 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:47.438309 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:47.438309 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:47.438309 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:47.438309 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:47.438309 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:47.438309 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:47.438309 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:47.438309 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:47.438309 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:47.438309 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:47.438309 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:47.438309 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:47.438309 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:47.438309 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:47.438309 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:47.438309 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:47.439206 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:47.439206 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:47.439206 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:47.439206 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:47.442699 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 326/672] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:47.443893 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767081e-485f-4204-b4da-ae1029aad211 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:47.462457 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fca79d4-53f3-4a8e-9a48-8e49b0ed91d2 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:47.515878 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bba4b4-8698-4beb-aada-16a51df85859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:47.517592 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bba4b4-8698-4beb-aada-16a51df85859 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:47.518159 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bba4b4-8698-4beb-aada-16a51df85859 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:47.518159 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bba4b4-8698-4beb-aada-16a51df85859 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:47.531242 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bba4b4-8698-4beb-aada-16a51df85859 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:47.532739 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bba4b4-8698-4beb-aada-16a51df85859 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:47.544402 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:47.556559 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-a1bba4b4-8698-4beb-aada-16a51df85859 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:47.556559 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:47.556559 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:47.556559 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:47.556559 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:47.556559 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:47.556559 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:47.556559 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:47.556559 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:47.556559 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:47.556559 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:47.556559 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:47.556559 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:47.556559 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:47.556559 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:47.557531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:47.557531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:47.557531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:47.557531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:47.557531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:47.557531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:47.557531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:47.557531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:47.557531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:47.557531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:47.557531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:47.557531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:47.557531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:47.557531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:47.557531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:47.557531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:47.557531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:47.558435 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:47.558435 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:47.558435 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:47.558435 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:47.558435 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 347/673] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:47.558711 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bba4b4-8698-4beb-aada-16a51df85859 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:47.572772 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767081e-485f-4204-b4da-ae1029aad211 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:47.619882 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bafe24-0b3f-4db0-b7ef-22b03872c8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:47.621077 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bafe24-0b3f-4db0-b7ef-22b03872c8be None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:47.621506 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bafe24-0b3f-4db0-b7ef-22b03872c8be None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:47.621950 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bafe24-0b3f-4db0-b7ef-22b03872c8be None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:47.623087 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bafe24-0b3f-4db0-b7ef-22b03872c8be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:47.623592 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bafe24-0b3f-4db0-b7ef-22b03872c8be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:47.626042 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bafe24-0b3f-4db0-b7ef-22b03872c8be None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:47.642092 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 327/674] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:47.643748 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bafe24-0b3f-4db0-b7ef-22b03872c8be None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:47.667910 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:47.694743 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bba4b4-8698-4beb-aada-16a51df85859 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:47.741062 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81d4707-0560-4d2c-83b8-e11616bfe283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:47.741943 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d4707-0560-4d2c-83b8-e11616bfe283 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:47.742317 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d4707-0560-4d2c-83b8-e11616bfe283 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:47.742690 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d4707-0560-4d2c-83b8-e11616bfe283 None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:47.743863 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81d4707-0560-4d2c-83b8-e11616bfe283 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:47.744186 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81d4707-0560-4d2c-83b8-e11616bfe283 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:47.745569 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81d4707-0560-4d2c-83b8-e11616bfe283 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:47.761311 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 348/675] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:47.763376 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81d4707-0560-4d2c-83b8-e11616bfe283 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:47.781434 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bafe24-0b3f-4db0-b7ef-22b03872c8be None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:47.826625 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66046e29-5851-4e6f-9303-12161a3494a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:47.827704 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66046e29-5851-4e6f-9303-12161a3494a2 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:47.828205 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66046e29-5851-4e6f-9303-12161a3494a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:47.828729 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66046e29-5851-4e6f-9303-12161a3494a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:47.829695 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66046e29-5851-4e6f-9303-12161a3494a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:47.830204 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66046e29-5851-4e6f-9303-12161a3494a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:47.831855 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66046e29-5851-4e6f-9303-12161a3494a2 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:47.841014 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:47.846222 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 328/676] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:47.847385 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66046e29-5851-4e6f-9303-12161a3494a2 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:47.894624 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81d4707-0560-4d2c-83b8-e11616bfe283 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:47.929432 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:47.932967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca465f-c346-47c3-8e36-ab89d35472ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:47.933666 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca465f-c346-47c3-8e36-ab89d35472ec None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:47.939134 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca465f-c346-47c3-8e36-ab89d35472ec None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:47.939134 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca465f-c346-47c3-8e36-ab89d35472ec None admin] PATH_INFO: `/v3/users/swift` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:47.942346 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca465f-c346-47c3-8e36-ab89d35472ec None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:47.943819 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca465f-c346-47c3-8e36-ab89d35472ec None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:47.956864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66046e29-5851-4e6f-9303-12161a3494a2 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:47.962480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-03ca465f-c346-47c3-8e36-ab89d35472ec None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 07 15:15:47.962480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:47.962480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:47.962480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:47.962480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:47.962480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:47.962480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:47.962480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:47.962480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:47.962480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:47.962480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:47.962480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:47.962480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 15:15:47.962480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 15:15:47.962480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 15:15:47.962480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 15:15:47.963424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:47.963424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:47.963424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 15:15:47.963424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:15:47.963424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 15:15:47.963424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:15:47.963424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:47.963424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:47.963424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:47.963424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:47.963424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:47.963424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:47.963424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:47.963424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:47.963424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:47.963424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:47.963424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:47.963424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:47.964411 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:47.964411 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:47.964411 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 15:15:47.964411 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 15:15:47.964411 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 15:15:47.964411 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 15:15:47.964411 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 15:15:47.964411 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 15:15:47.964411 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 07 15:15:47.964411 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:47.968975 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 349/677] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Jul 7 15:15:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:47.969851 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca465f-c346-47c3-8e36-ab89d35472ec None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:47.996041 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a727c25-5846-49f0-90cd-45230eb11972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:47.997017 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a727c25-5846-49f0-90cd-45230eb11972 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:47.997395 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a727c25-5846-49f0-90cd-45230eb11972 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:47.997761 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a727c25-5846-49f0-90cd-45230eb11972 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:48.004970 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a727c25-5846-49f0-90cd-45230eb11972 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:48.006567 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a727c25-5846-49f0-90cd-45230eb11972 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:48.021064 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-8a727c25-5846-49f0-90cd-45230eb11972 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:15:48.021064 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:48.021064 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:48.021064 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:48.021064 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:48.021064 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:48.021064 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:48.021064 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:48.021064 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:48.021064 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:48.021064 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:48.021064 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:48.021064 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 15:15:48.021064 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 15:15:48.021064 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 15:15:48.022112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 15:15:48.022112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:48.022112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:48.022112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:48.022112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:48.022112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:48.022112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:48.022112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:48.022112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:48.022112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:48.022112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:48.022112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:48.022112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:48.022112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:48.022112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:48.022112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:48.022112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:48.023057 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 15:15:48.023057 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 15:15:48.023057 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 15:15:48.023057 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 15:15:48.023057 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 15:15:48.023057 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 15:15:48.023057 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:15:48.023057 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:48.023057 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 329/678] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:15:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:48.023552 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a727c25-5846-49f0-90cd-45230eb11972 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:48.081185 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca465f-c346-47c3-8e36-ab89d35472ec None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:48.103670 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:48.118729 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8471c97d-c0a3-4ac9-bbbb-76bb85ab3cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:48.120442 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471c97d-c0a3-4ac9-bbbb-76bb85ab3cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:48.120442 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471c97d-c0a3-4ac9-bbbb-76bb85ab3cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:48.120442 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471c97d-c0a3-4ac9-bbbb-76bb85ab3cd3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:48.127018 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a727c25-5846-49f0-90cd-45230eb11972 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:48.127699 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8471c97d-c0a3-4ac9-bbbb-76bb85ab3cd3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:48.129382 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8471c97d-c0a3-4ac9-bbbb-76bb85ab3cd3 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:48.145163 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-8471c97d-c0a3-4ac9-bbbb-76bb85ab3cd3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:15:48.145163 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:48.145163 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:48.145163 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:48.145163 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:48.145163 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:48.145163 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:48.145163 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:48.145163 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:48.145163 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:48.145163 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:48.145163 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:48.145163 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 15:15:48.145163 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 15:15:48.145163 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 15:15:48.146256 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 15:15:48.146256 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:48.146256 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:48.146256 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:48.146256 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:48.146256 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:48.146256 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:48.146256 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:48.146256 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:48.146256 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:48.146256 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:48.146256 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:48.146256 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:48.146256 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:48.146256 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:48.146256 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:48.146256 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:48.147185 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 15:15:48.147185 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 15:15:48.147185 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 15:15:48.147185 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 15:15:48.147185 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 15:15:48.147185 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 15:15:48.147185 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:15:48.147185 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:48.151385 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 350/679] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:15:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:48.152720 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c30ac8-5ab8-4335-9068-5b06f2498036 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:48.153581 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c30ac8-5ab8-4335-9068-5b06f2498036 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:48.153866 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c30ac8-5ab8-4335-9068-5b06f2498036 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:48.155181 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 351/680] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:48.156530 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c30ac8-5ab8-4335-9068-5b06f2498036 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:48.169756 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d45892b-7106-4773-bd11-664f2ae78665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:48.170418 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45892b-7106-4773-bd11-664f2ae78665 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:48.170721 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45892b-7106-4773-bd11-664f2ae78665 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:48.171020 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45892b-7106-4773-bd11-664f2ae78665 None admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:48.171889 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d45892b-7106-4773-bd11-664f2ae78665 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:48.172195 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d45892b-7106-4773-bd11-664f2ae78665 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:48.173595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d45892b-7106-4773-bd11-664f2ae78665 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:48.175407 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:48.197114 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 330/681] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Jul 7 15:15:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:48.197114 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d45892b-7106-4773-bd11-664f2ae78665 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:48.268186 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c30ac8-5ab8-4335-9068-5b06f2498036 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:48.301364 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb131c-d0b2-48d6-a678-ff6b3a4551e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:48.302012 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb131c-d0b2-48d6-a678-ff6b3a4551e8 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:48.302286 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb131c-d0b2-48d6-a678-ff6b3a4551e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:48.302563 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb131c-d0b2-48d6-a678-ff6b3a4551e8 None admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:48.303332 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cb131c-d0b2-48d6-a678-ff6b3a4551e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:48.303664 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cb131c-d0b2-48d6-a678-ff6b3a4551e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:48.305088 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cb131c-d0b2-48d6-a678-ff6b3a4551e8 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:48.306609 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d45892b-7106-4773-bd11-664f2ae78665 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:48.316119 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 352/682] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Jul 7 15:15:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:48.317276 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085c779-5ee7-4318-9fce-e4cfb6bfe9f9 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:48.317557 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085c779-5ee7-4318-9fce-e4cfb6bfe9f9 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:48.317832 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085c779-5ee7-4318-9fce-e4cfb6bfe9f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:48.333935 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-0085c779-5ee7-4318-9fce-e4cfb6bfe9f9 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:48.335104 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-0085c779-5ee7-4318-9fce-e4cfb6bfe9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:48.350009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad95a0-e28d-454d-a2f9-6749f114706f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:48.350886 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad95a0-e28d-454d-a2f9-6749f114706f None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:48.351337 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad95a0-e28d-454d-a2f9-6749f114706f None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:48.351848 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad95a0-e28d-454d-a2f9-6749f114706f None admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:48.352916 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ad95a0-e28d-454d-a2f9-6749f114706f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:48.353392 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ad95a0-e28d-454d-a2f9-6749f114706f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:48.355061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ad95a0-e28d-454d-a2f9-6749f114706f None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:48.367863 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 331/683] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Jul 7 15:15:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:48.368816 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad95a0-e28d-454d-a2f9-6749f114706f None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:48.374488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:48.406897 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 353/684] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:48.407913 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085c779-5ee7-4318-9fce-e4cfb6bfe9f9 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:48.443743 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:48.494235 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad95a0-e28d-454d-a2f9-6749f114706f None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:48.510654 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085c779-5ee7-4318-9fce-e4cfb6bfe9f9 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:48.529634 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272755a7-cf06-478d-b174-84f14dbbe977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:48.530345 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272755a7-cf06-478d-b174-84f14dbbe977 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:48.530628 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272755a7-cf06-478d-b174-84f14dbbe977 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:48.530909 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272755a7-cf06-478d-b174-84f14dbbe977 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:48.549786 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0bcbe0-53d2-42b6-8ab9-8c7ff11e25c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:48.550653 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bcbe0-53d2-42b6-8ab9-8c7ff11e25c8 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:48.550764 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bcbe0-53d2-42b6-8ab9-8c7ff11e25c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:48.550972 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bcbe0-53d2-42b6-8ab9-8c7ff11e25c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:48.551832 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0bcbe0-53d2-42b6-8ab9-8c7ff11e25c8 None admin] RBAC: Adding query filter params (role.id=3d4f4e0a472e4ff9ac68d90c8612e956, scope.project.id=c5260c7fd3614e9e93b167c86a94260e, user.id=2a0fbebb3dd2458ba4fe5d7f0ce211b8) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:48.552142 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0bcbe0-53d2-42b6-8ab9-8c7ff11e25c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:48.553568 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0bcbe0-53d2-42b6-8ab9-8c7ff11e25c8 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:48.560003 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 354/685] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:15:48 2022] GET /identity/v3/role_assignments?user.id=2a0fbebb3dd2458ba4fe5d7f0ce211b8&scope.project.id=c5260c7fd3614e9e93b167c86a94260e&role.id=3d4f4e0a472e4ff9ac68d90c8612e956 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:48.565257 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0bcbe0-53d2-42b6-8ab9-8c7ff11e25c8 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:48.610374 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272755a7-cf06-478d-b174-84f14dbbe977 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:48.611886 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272755a7-cf06-478d-b174-84f14dbbe977 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:48.624833 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:48.688409 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:48.700363 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0bcbe0-53d2-42b6-8ab9-8c7ff11e25c8 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:48.746738 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c583f9-a72f-46fc-b3fc-a1d96c3033d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:48.748275 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c583f9-a72f-46fc-b3fc-a1d96c3033d0 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:48.748556 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c583f9-a72f-46fc-b3fc-a1d96c3033d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:48.748829 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c583f9-a72f-46fc-b3fc-a1d96c3033d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:48.750210 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c583f9-a72f-46fc-b3fc-a1d96c3033d0 None admin] RBAC: Adding query filter params (role.id=3d4f4e0a472e4ff9ac68d90c8612e956, scope.project.id=c5260c7fd3614e9e93b167c86a94260e, user.id=e79efaea155c46ecb9831956e68aec8e) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:48.750517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c583f9-a72f-46fc-b3fc-a1d96c3033d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:48.752367 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c583f9-a72f-46fc-b3fc-a1d96c3033d0 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:48.762722 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 355/686] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:15:48 2022] GET /identity/v3/role_assignments?user.id=e79efaea155c46ecb9831956e68aec8e&scope.project.id=c5260c7fd3614e9e93b167c86a94260e&role.id=3d4f4e0a472e4ff9ac68d90c8612e956 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:48.763380 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 332/687] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:15:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 15:15:48.763466 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c583f9-a72f-46fc-b3fc-a1d96c3033d0 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:48.767776 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272755a7-cf06-478d-b174-84f14dbbe977 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:48.898766 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272755a7-cf06-478d-b174-84f14dbbe977 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:48.916878 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:48.922157 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c583f9-a72f-46fc-b3fc-a1d96c3033d0 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:48.940752 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1773b871-af72-495f-8b30-1f866fc3ee21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:48.941851 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773b871-af72-495f-8b30-1f866fc3ee21 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:48.942134 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773b871-af72-495f-8b30-1f866fc3ee21 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:48.942407 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773b871-af72-495f-8b30-1f866fc3ee21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:48.957384 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1773b871-af72-495f-8b30-1f866fc3ee21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:48.959193 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1773b871-af72-495f-8b30-1f866fc3ee21 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:48.968557 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc78e215-597a-41f0-b0d4-47ac45d4d53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:48.969597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78e215-597a-41f0-b0d4-47ac45d4d53c None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:48.970095 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78e215-597a-41f0-b0d4-47ac45d4d53c None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:48.970583 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78e215-597a-41f0-b0d4-47ac45d4d53c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:48.981485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-1773b871-af72-495f-8b30-1f866fc3ee21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:48.981485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:48.981485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:48.981485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:48.981485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:48.981485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:48.981485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:48.981485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:48.981485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:48.981485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:48.981485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:48.981485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:48.981485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:48.981485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:48.981485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:48.982424 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:48.982424 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:48.982424 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:48.982424 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:48.982424 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:48.982424 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:48.982424 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:48.982424 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:48.982424 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:48.982424 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:48.982424 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:48.982424 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:48.982424 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:48.982424 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:48.982424 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:48.982424 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:48.982424 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:48.984418 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:48.984418 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:48.984418 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:48.984418 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:48.984418 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 333/688] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:15:48.986021 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc78e215-597a-41f0-b0d4-47ac45d4d53c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:48.989767 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1773b871-af72-495f-8b30-1f866fc3ee21 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:48.997150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-bc78e215-597a-41f0-b0d4-47ac45d4d53c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:15:48.997150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:48.997150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:48.997150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:48.997150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:48.997150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:48.997150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:48.997150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:48.997150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:48.997150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:48.997150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:48.997150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:48.997150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:48.997150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:48.997150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:48.997150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:48.998118 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:48.998118 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:48.998118 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:48.998118 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:48.998118 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:48.998118 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:48.998118 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:48.998118 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:48.998118 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:48.998118 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:48.998118 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:48.998118 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:48.998118 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:48.998118 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:48.998118 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:48.998118 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:48.999596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:48.999596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:48.999596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:48.999596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:48.999596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:48.999596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:48.999596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:48.999596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:48.999596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:48.999596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:48.999596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:15:48.999596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:48.999596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 356/689] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Jul 7 15:15:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:49.005350 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc78e215-597a-41f0-b0d4-47ac45d4d53c None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:49.007913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:49.135841 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1773b871-af72-495f-8b30-1f866fc3ee21 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:49.138229 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc78e215-597a-41f0-b0d4-47ac45d4d53c None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:49.177257 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248dc77-3aa2-4af8-80d1-9302228c2c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:49.177897 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248dc77-3aa2-4af8-80d1-9302228c2c41 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:49.178243 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248dc77-3aa2-4af8-80d1-9302228c2c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:49.178636 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248dc77-3aa2-4af8-80d1-9302228c2c41 None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:49.179395 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b248dc77-3aa2-4af8-80d1-9302228c2c41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:49.179896 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b248dc77-3aa2-4af8-80d1-9302228c2c41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:49.181589 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b248dc77-3aa2-4af8-80d1-9302228c2c41 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:49.183712 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd95f3-86de-48c6-b97b-7704f39bf49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:49.185043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd95f3-86de-48c6-b97b-7704f39bf49e None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:49.185508 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd95f3-86de-48c6-b97b-7704f39bf49e None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:49.185969 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd95f3-86de-48c6-b97b-7704f39bf49e None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:49.186964 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bd95f3-86de-48c6-b97b-7704f39bf49e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:49.187451 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bd95f3-86de-48c6-b97b-7704f39bf49e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:49.189517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bd95f3-86de-48c6-b97b-7704f39bf49e None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:49.194595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 334/690] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:49.199932 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 357/691] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:15:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:49.204105 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248dc77-3aa2-4af8-80d1-9302228c2c41 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:49.205077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd95f3-86de-48c6-b97b-7704f39bf49e None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:49.252433 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:49.340769 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd95f3-86de-48c6-b97b-7704f39bf49e None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:49.360025 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:49.378292 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79088ac2-ba29-4cde-b615-691b538b51e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:49.379410 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79088ac2-ba29-4cde-b615-691b538b51e4 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:49.379874 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79088ac2-ba29-4cde-b615-691b538b51e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:49.382087 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79088ac2-ba29-4cde-b615-691b538b51e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:49.406900 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79088ac2-ba29-4cde-b615-691b538b51e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:49.409233 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79088ac2-ba29-4cde-b615-691b538b51e4 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:49.425352 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248dc77-3aa2-4af8-80d1-9302228c2c41 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:49.441597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-79088ac2-ba29-4cde-b615-691b538b51e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:49.441597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:49.441597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:49.441597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:49.441597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:49.441597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:49.441597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:49.441597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:49.441597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:49.441597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:49.441597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:49.441597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:49.441597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:49.441597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:49.441597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:49.442789 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:49.442789 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:49.442789 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:49.442789 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:49.442789 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:49.442789 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:49.442789 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:49.442789 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:49.442789 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:49.442789 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:49.442789 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:49.442789 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:49.442789 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:49.442789 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:49.442789 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:49.442789 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:49.442789 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:49.443926 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:49.443926 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:49.443926 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:49.443926 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:49.449328 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 358/692] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:49.454319 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79088ac2-ba29-4cde-b615-691b538b51e4 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:49.486839 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ff282-650d-4b15-804c-5a841b2e8425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:49.492144 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ff282-650d-4b15-804c-5a841b2e8425 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:49.492271 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ff282-650d-4b15-804c-5a841b2e8425 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:49.492540 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ff282-650d-4b15-804c-5a841b2e8425 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:49.504584 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24ff282-650d-4b15-804c-5a841b2e8425 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:49.506075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24ff282-650d-4b15-804c-5a841b2e8425 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:49.522076 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-b24ff282-650d-4b15-804c-5a841b2e8425 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:15:49.522076 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:49.522076 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:49.522076 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:49.522076 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:49.522076 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:49.522076 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:49.522076 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:49.522076 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:49.522076 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:49.522076 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:49.522076 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:49.522076 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 15:15:49.522076 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 15:15:49.522076 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 15:15:49.523029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 15:15:49.523029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:49.523029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:49.523029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:49.523029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:49.523029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:49.523029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:49.523029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:49.523029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:49.523029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:49.523029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:49.523029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:49.523029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:49.523029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:49.523029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:49.523029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:49.523029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:49.523949 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 15:15:49.523949 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 15:15:49.523949 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 15:15:49.523949 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 15:15:49.523949 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 15:15:49.523949 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 15:15:49.523949 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:15:49.523949 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:49.523949 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 335/693] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:15:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:49.529800 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ff282-650d-4b15-804c-5a841b2e8425 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:49.541946 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:49.565905 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79088ac2-ba29-4cde-b615-691b538b51e4 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:49.606070 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a8214f-28c0-4443-9281-5c508606195f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:49.607077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a8214f-28c0-4443-9281-5c508606195f None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:49.608221 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a8214f-28c0-4443-9281-5c508606195f None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:49.608539 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a8214f-28c0-4443-9281-5c508606195f None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:49.609285 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a8214f-28c0-4443-9281-5c508606195f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:49.609596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a8214f-28c0-4443-9281-5c508606195f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:49.610982 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a8214f-28c0-4443-9281-5c508606195f None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:49.627910 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 359/694] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:49.629527 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d637b-44b5-4ab6-ad72-506ba43add9f None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:49.629814 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d637b-44b5-4ab6-ad72-506ba43add9f None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:49.630092 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d637b-44b5-4ab6-ad72-506ba43add9f None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:49.635629 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 360/695] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:49.635629 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d637b-44b5-4ab6-ad72-506ba43add9f None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:49.663958 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ff282-650d-4b15-804c-5a841b2e8425 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:49.695404 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:49.711701 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f82a5-d06f-4d3e-b995-76e16a74be08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:49.712876 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f82a5-d06f-4d3e-b995-76e16a74be08 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:49.713686 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f82a5-d06f-4d3e-b995-76e16a74be08 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:49.714385 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f82a5-d06f-4d3e-b995-76e16a74be08 None admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:49.715388 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f82a5-d06f-4d3e-b995-76e16a74be08 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:49.725967 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f82a5-d06f-4d3e-b995-76e16a74be08 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:49.725967 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f82a5-d06f-4d3e-b995-76e16a74be08 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:49.736085 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 336/696] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Jul 7 15:15:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:49.738362 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1daccb0-e2f2-498e-ba45-7f69bcecaf98 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:49.738895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1daccb0-e2f2-498e-ba45-7f69bcecaf98 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:49.739386 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1daccb0-e2f2-498e-ba45-7f69bcecaf98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:49.755779 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d637b-44b5-4ab6-ad72-506ba43add9f None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:49.761945 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-b1daccb0-e2f2-498e-ba45-7f69bcecaf98 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:49.764021 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-b1daccb0-e2f2-498e-ba45-7f69bcecaf98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:49.793260 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:49.798243 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1f69e-28b5-46a8-b627-250a03343363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:49.799011 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1f69e-28b5-46a8-b627-250a03343363 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:49.799294 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1f69e-28b5-46a8-b627-250a03343363 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:49.799585 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1f69e-28b5-46a8-b627-250a03343363 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:49.806957 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a1f69e-28b5-46a8-b627-250a03343363 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:49.808540 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a1f69e-28b5-46a8-b627-250a03343363 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:49.823154 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-39a1f69e-28b5-46a8-b627-250a03343363 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 07 15:15:49.823154 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:49.823154 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:49.823154 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:49.823154 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:49.823154 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:49.823154 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:49.823154 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:49.823154 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:49.823154 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:49.823154 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:49.823154 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:49.823154 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 15:15:49.823154 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 15:15:49.823154 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 15:15:49.823154 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 15:15:49.824139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:49.824139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:49.824139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 15:15:49.824139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:15:49.824139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 15:15:49.824139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:15:49.824139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:49.824139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:49.824139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:49.824139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:49.824139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:49.824139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:49.824139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:49.824139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:49.824139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:49.824139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:49.824139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:49.824139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:49.825124 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:49.825124 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:49.825124 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 15:15:49.825124 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 15:15:49.825124 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 15:15:49.825124 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 15:15:49.825124 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 15:15:49.825124 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 15:15:49.825124 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 07 15:15:49.825124 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:49.825124 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 361/697] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Jul 7 15:15:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:49.826299 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1f69e-28b5-46a8-b627-250a03343363 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:49.872060 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 337/698] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:49.874646 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1daccb0-e2f2-498e-ba45-7f69bcecaf98 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:49.979206 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1f69e-28b5-46a8-b627-250a03343363 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:49.997960 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1daccb0-e2f2-498e-ba45-7f69bcecaf98 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:50.013317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:50.018432 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3114412-8475-4d27-b31c-78f6c83f21b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:50.019347 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3114412-8475-4d27-b31c-78f6c83f21b8 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:50.019865 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3114412-8475-4d27-b31c-78f6c83f21b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:50.020339 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3114412-8475-4d27-b31c-78f6c83f21b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:50.021375 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3114412-8475-4d27-b31c-78f6c83f21b8 None admin] RBAC: Adding query filter params (role.id=3d4f4e0a472e4ff9ac68d90c8612e956, scope.project.id=c5260c7fd3614e9e93b167c86a94260e, user.id=b18033f849b240999d12c1885d2f7b91) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:50.021923 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3114412-8475-4d27-b31c-78f6c83f21b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:50.023554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3114412-8475-4d27-b31c-78f6c83f21b8 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:50.030866 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 362/699] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:15:49 2022] GET /identity/v3/role_assignments?user.id=b18033f849b240999d12c1885d2f7b91&scope.project.id=c5260c7fd3614e9e93b167c86a94260e&role.id=3d4f4e0a472e4ff9ac68d90c8612e956 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:50.032071 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3114412-8475-4d27-b31c-78f6c83f21b8 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:50.035241 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675fa683-01be-478e-836b-466903563ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:50.035993 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675fa683-01be-478e-836b-466903563ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:50.036283 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675fa683-01be-478e-836b-466903563ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:50.036578 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675fa683-01be-478e-836b-466903563ed1 None admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:50.037749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675fa683-01be-478e-836b-466903563ed1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:50.037869 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675fa683-01be-478e-836b-466903563ed1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:50.039265 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675fa683-01be-478e-836b-466903563ed1 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:50.061063 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 338/700] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Jul 7 15:15:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:50.067224 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675fa683-01be-478e-836b-466903563ed1 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:50.099935 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:50.152333 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3114412-8475-4d27-b31c-78f6c83f21b8 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:50.195154 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf74274-f072-4f5d-8596-726c84220482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:50.196365 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf74274-f072-4f5d-8596-726c84220482 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:50.196852 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf74274-f072-4f5d-8596-726c84220482 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:50.197328 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf74274-f072-4f5d-8596-726c84220482 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:50.209903 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf74274-f072-4f5d-8596-726c84220482 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:50.211786 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf74274-f072-4f5d-8596-726c84220482 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:50.230887 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-8cf74274-f072-4f5d-8596-726c84220482 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:50.230887 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:50.230887 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:50.230887 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:50.230887 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:50.230887 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:50.230887 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:50.230887 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:50.230887 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:50.230887 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:50.230887 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:50.230887 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:50.230887 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:50.230887 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:50.230887 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:50.231907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:50.231907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:50.231907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:50.231907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:50.231907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:50.231907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:50.231907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:50.231907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:50.231907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:50.231907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:50.231907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:50.231907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:50.231907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:50.231907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:50.231907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:50.231907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:50.231907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:50.232816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:50.232816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:50.232816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:50.232816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:50.235558 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 363/701] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:50.241317 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf74274-f072-4f5d-8596-726c84220482 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:50.259808 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675fa683-01be-478e-836b-466903563ed1 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:50.298970 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e81b0e6-17eb-485b-b373-15ff1ac46350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:50.299598 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81b0e6-17eb-485b-b373-15ff1ac46350 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:50.299888 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81b0e6-17eb-485b-b373-15ff1ac46350 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:50.300166 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81b0e6-17eb-485b-b373-15ff1ac46350 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:50.346497 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf74274-f072-4f5d-8596-726c84220482 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:50.348021 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:50.356851 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:50.378608 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e81b0e6-17eb-485b-b373-15ff1ac46350 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:50.380128 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e81b0e6-17eb-485b-b373-15ff1ac46350 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:50.384187 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1578e5d-16f6-4083-9d6c-f35b4aec180f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:50.385078 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1578e5d-16f6-4083-9d6c-f35b4aec180f None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:50.385555 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1578e5d-16f6-4083-9d6c-f35b4aec180f None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:50.386023 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1578e5d-16f6-4083-9d6c-f35b4aec180f None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:50.386932 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1578e5d-16f6-4083-9d6c-f35b4aec180f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:50.387447 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1578e5d-16f6-4083-9d6c-f35b4aec180f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:50.389656 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1578e5d-16f6-4083-9d6c-f35b4aec180f None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:50.402167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 364/702] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:15:50.408652 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1578e5d-16f6-4083-9d6c-f35b4aec180f None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:50.526599 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1578e5d-16f6-4083-9d6c-f35b4aec180f None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:50.531728 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 339/703] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:15:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 07 15:15:50.532987 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e81b0e6-17eb-485b-b373-15ff1ac46350 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:50.569964 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955a97c-f739-41c4-b7cc-2ea3550f48fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:50.571469 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955a97c-f739-41c4-b7cc-2ea3550f48fa None admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:50.572234 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955a97c-f739-41c4-b7cc-2ea3550f48fa None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:50.573016 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955a97c-f739-41c4-b7cc-2ea3550f48fa None admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:50.574840 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b955a97c-f739-41c4-b7cc-2ea3550f48fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:50.577749 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b955a97c-f739-41c4-b7cc-2ea3550f48fa None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:50.642532 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:50.681507 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 365/704] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Jul 7 15:15:50 2022] POST /identity/v3/users => generated 254 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:15:50.682849 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e81b0e6-17eb-485b-b373-15ff1ac46350 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:50.740537 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8fc478-2fb6-4eab-bd8b-da9b987956d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:50.741794 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fc478-2fb6-4eab-bd8b-da9b987956d2 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:50.742285 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fc478-2fb6-4eab-bd8b-da9b987956d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:50.742861 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fc478-2fb6-4eab-bd8b-da9b987956d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:50.758491 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8fc478-2fb6-4eab-bd8b-da9b987956d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:50.760429 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8fc478-2fb6-4eab-bd8b-da9b987956d2 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:50.787980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-be8fc478-2fb6-4eab-bd8b-da9b987956d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:50.787980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:50.787980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:50.787980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:50.787980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:50.787980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:50.787980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:50.787980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:50.787980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:50.787980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:50.787980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:50.787980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:50.787980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:50.787980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:50.787980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:50.788966 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:50.788966 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:50.788966 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:50.788966 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:50.788966 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:50.788966 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:50.788966 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:50.788966 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:50.788966 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:50.788966 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:50.788966 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:50.788966 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:50.788966 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:50.788966 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:50.788966 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:50.788966 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:50.788966 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:50.789874 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:50.789874 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:50.789874 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:50.789874 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:50.791931 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 340/705] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:50.797571 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955a97c-f739-41c4-b7cc-2ea3550f48fa None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:50.836963 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:50.929546 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955a97c-f739-41c4-b7cc-2ea3550f48fa None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:50.975279 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67496bb-1af3-476c-bdf0-832864aff3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:50.976057 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67496bb-1af3-476c-bdf0-832864aff3a9 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:50.976341 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67496bb-1af3-476c-bdf0-832864aff3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:50.976614 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67496bb-1af3-476c-bdf0-832864aff3a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:50.977351 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67496bb-1af3-476c-bdf0-832864aff3a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:50.977660 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67496bb-1af3-476c-bdf0-832864aff3a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:50.979029 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67496bb-1af3-476c-bdf0-832864aff3a9 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:50.994493 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 366/706] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:51.000641 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8fc478-2fb6-4eab-bd8b-da9b987956d2 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:51.126094 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8fc478-2fb6-4eab-bd8b-da9b987956d2 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:51.156183 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:51.165232 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea97f2-2e63-4a60-a872-1f9bbc05381f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:51.173406 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea97f2-2e63-4a60-a872-1f9bbc05381f None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:51.173406 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea97f2-2e63-4a60-a872-1f9bbc05381f None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:51.173406 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea97f2-2e63-4a60-a872-1f9bbc05381f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:51.178641 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea97f2-2e63-4a60-a872-1f9bbc05381f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:51.180198 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea97f2-2e63-4a60-a872-1f9bbc05381f None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:51.204440 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-78ea97f2-2e63-4a60-a872-1f9bbc05381f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:15:51.204440 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:51.204440 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:51.204440 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:51.204440 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:51.204440 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:51.204440 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:51.204440 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:51.204440 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:51.204440 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:51.204440 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:51.204440 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:51.204440 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 15:15:51.204440 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 15:15:51.204440 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 15:15:51.205407 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 15:15:51.205407 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:51.205407 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:51.205407 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:51.205407 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:51.205407 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:51.205407 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:51.205407 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:51.205407 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:51.205407 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:51.205407 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:51.205407 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:51.205407 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:51.205407 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:51.205407 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:51.205407 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:51.205407 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:51.206316 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 15:15:51.206316 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 15:15:51.206316 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 15:15:51.206316 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 15:15:51.206316 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 15:15:51.206316 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 15:15:51.206316 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:15:51.206316 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:51.206316 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 341/707] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:15:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:51.212970 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67496bb-1af3-476c-bdf0-832864aff3a9 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:51.269925 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aac071-463f-4a61-9c35-9073af080173 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:51.270362 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aac071-463f-4a61-9c35-9073af080173 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:51.270641 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aac071-463f-4a61-9c35-9073af080173 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:51.271841 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 342/708] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:15:51.278620 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca1605-4ba3-440d-845a-f5d174cdd8c0 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:51.278885 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca1605-4ba3-440d-845a-f5d174cdd8c0 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:51.279289 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca1605-4ba3-440d-845a-f5d174cdd8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:51.301173 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-66ca1605-4ba3-440d-845a-f5d174cdd8c0 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:51.302387 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-66ca1605-4ba3-440d-845a-f5d174cdd8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:51.358904 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:51.377905 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67496bb-1af3-476c-bdf0-832864aff3a9 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:51.415313 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 343/709] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:51.417383 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd2fdc-8b15-47b3-ba84-3689238344dc None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:51.417754 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd2fdc-8b15-47b3-ba84-3689238344dc None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:51.418102 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd2fdc-8b15-47b3-ba84-3689238344dc None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:51.420370 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 344/710] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:51.426490 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd796a-261e-42c7-9a48-6b2685027a4b None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:51.426791 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd796a-261e-42c7-9a48-6b2685027a4b None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:51.427060 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd796a-261e-42c7-9a48-6b2685027a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:51.427599 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455033db-8c75-48b2-9ef2-9a0d2d7eba3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:51.428567 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455033db-8c75-48b2-9ef2-9a0d2d7eba3a None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:51.429026 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455033db-8c75-48b2-9ef2-9a0d2d7eba3a None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:51.429475 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455033db-8c75-48b2-9ef2-9a0d2d7eba3a None admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:51.430542 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455033db-8c75-48b2-9ef2-9a0d2d7eba3a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:51.431060 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455033db-8c75-48b2-9ef2-9a0d2d7eba3a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:51.433857 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455033db-8c75-48b2-9ef2-9a0d2d7eba3a None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:51.445542 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-5edd796a-261e-42c7-9a48-6b2685027a4b None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:51.446784 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-5edd796a-261e-42c7-9a48-6b2685027a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:51.448148 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 367/711] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Jul 7 15:15:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:51.449540 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455033db-8c75-48b2-9ef2-9a0d2d7eba3a None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:51.531941 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 345/712] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 15:15:51.535976 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd796a-261e-42c7-9a48-6b2685027a4b None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:51.590519 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455033db-8c75-48b2-9ef2-9a0d2d7eba3a None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:51.601691 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:51.631916 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb82ad4-f186-4e30-948d-2abcb76bdb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:51.633380 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb82ad4-f186-4e30-948d-2abcb76bdb46 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:51.633380 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb82ad4-f186-4e30-948d-2abcb76bdb46 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:51.633380 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb82ad4-f186-4e30-948d-2abcb76bdb46 None admin] PATH_INFO: `/v3/services/network` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:51.633857 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb82ad4-f186-4e30-948d-2abcb76bdb46 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:51.635285 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb82ad4-f186-4e30-948d-2abcb76bdb46 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:51.649995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-0cb82ad4-f186-4e30-948d-2abcb76bdb46 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 07 15:15:51.649995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:51.649995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 15:15:51.649995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 15:15:51.649995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 15:15:51.649995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 15:15:51.649995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 15:15:51.649995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:15:51.649995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 07 15:15:51.649995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:51.649995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 15:15:51.649995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:51.649995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:51.649995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:51.649995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:51.649995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:51.649995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:51.651191 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:51.651191 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:51.651191 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:51.651191 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:51.651191 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:51.651191 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:51.651191 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 15:15:51.651191 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 15:15:51.651191 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 15:15:51.651191 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 15:15:51.651191 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:51.651191 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:51.651191 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:51.651191 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:51.651191 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:51.651191 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:51.652136 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:51.652136 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:51.652136 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:51.652136 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:51.652136 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:51.652136 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:51.652136 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:51.652136 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:51.652136 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:51.652136 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:51.652136 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 15:15:51.652136 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:15:51.652136 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 07 15:15:51.652136 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:51.655730 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 368/713] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:15:51 2022] GET /identity/v3/services/network => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:15:51.656724 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb82ad4-f186-4e30-948d-2abcb76bdb46 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:51.674294 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:51.680056 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd796a-261e-42c7-9a48-6b2685027a4b None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:51.724890 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7f264-b058-4f5e-b03a-b995cc853f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:51.725989 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7f264-b058-4f5e-b03a-b995cc853f93 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:51.726786 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7f264-b058-4f5e-b03a-b995cc853f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:51.727549 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7f264-b058-4f5e-b03a-b995cc853f93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:51.728839 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b7f264-b058-4f5e-b03a-b995cc853f93 None admin] RBAC: Adding query filter params (role.id=0e7793fbf3064037984da651879e3a94, scope.project.id=c5260c7fd3614e9e93b167c86a94260e, user.id=de1632555cb345f2b9213da1925c02f4) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:51.729356 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b7f264-b058-4f5e-b03a-b995cc853f93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:51.731738 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b7f264-b058-4f5e-b03a-b995cc853f93 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:51.740750 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 346/714] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:15:51 2022] GET /identity/v3/role_assignments?user.id=de1632555cb345f2b9213da1925c02f4&scope.project.id=c5260c7fd3614e9e93b167c86a94260e&role.id=0e7793fbf3064037984da651879e3a94 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:51.742114 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7f264-b058-4f5e-b03a-b995cc853f93 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:51.805903 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb82ad4-f186-4e30-948d-2abcb76bdb46 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:51.880134 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaec5ad-5e7e-4b02-aefe-4c2196748ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:51.881038 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaec5ad-5e7e-4b02-aefe-4c2196748ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:51.881324 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaec5ad-5e7e-4b02-aefe-4c2196748ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:51.881597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaec5ad-5e7e-4b02-aefe-4c2196748ae4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:51.882317 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaec5ad-5e7e-4b02-aefe-4c2196748ae4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:51.883699 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaec5ad-5e7e-4b02-aefe-4c2196748ae4 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:51.903463 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-eaaec5ad-5e7e-4b02-aefe-4c2196748ae4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 07 15:15:51.903463 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:51.903463 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 15:15:51.903463 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 15:15:51.903463 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 15:15:51.903463 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 15:15:51.903463 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 15:15:51.903463 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:15:51.903463 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 07 15:15:51.903463 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:51.903463 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 15:15:51.903463 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:51.903463 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:51.903463 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:51.903463 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:51.903463 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:51.904504 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:51.904504 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:51.904504 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:51.904504 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:51.904504 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:51.904504 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:51.904504 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:51.904504 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 15:15:51.904504 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 15:15:51.904504 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 15:15:51.904504 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 15:15:51.904504 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:51.904504 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:51.904504 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:51.904504 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:51.904504 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:51.905396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:51.905396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:51.905396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:51.905396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:51.905396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:51.905396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:51.905396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:51.905396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:51.905396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:51.905396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:51.905396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:51.905396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 15:15:51.905396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:15:51.905396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 07 15:15:51.905396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:51.905396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 369/715] 10.222.0.25 () {56 vars in 1129 bytes} [Thu Jul 7 15:15:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:51.912437 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaec5ad-5e7e-4b02-aefe-4c2196748ae4 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:51.989582 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7f264-b058-4f5e-b03a-b995cc853f93 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:52.006838 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:52.052840 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45893e5d-1ca8-4506-a8c8-d53370706206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:52.054969 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45893e5d-1ca8-4506-a8c8-d53370706206 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:52.055258 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45893e5d-1ca8-4506-a8c8-d53370706206 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:52.055529 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45893e5d-1ca8-4506-a8c8-d53370706206 None admin] PATH_INFO: `/v3/services` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:52.056259 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45893e5d-1ca8-4506-a8c8-d53370706206 None admin] RBAC: Adding query filter params (name=network) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:52.056563 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45893e5d-1ca8-4506-a8c8-d53370706206 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:52.057906 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45893e5d-1ca8-4506-a8c8-d53370706206 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:52.068781 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 347/716] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Jul 7 15:15:51 2022] GET /identity/v3/services?name=network => generated 124 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:52.074546 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45893e5d-1ca8-4506-a8c8-d53370706206 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:52.084958 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:52.146769 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaec5ad-5e7e-4b02-aefe-4c2196748ae4 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:52.189709 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db65830-a816-4c7d-8bff-759eeed1a3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:52.190729 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db65830-a816-4c7d-8bff-759eeed1a3c4 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:52.191246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db65830-a816-4c7d-8bff-759eeed1a3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:52.191770 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db65830-a816-4c7d-8bff-759eeed1a3c4 None admin] PATH_INFO: `/v3/services` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:52.192687 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db65830-a816-4c7d-8bff-759eeed1a3c4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:52.193209 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db65830-a816-4c7d-8bff-759eeed1a3c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:52.194785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db65830-a816-4c7d-8bff-759eeed1a3c4 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:52.201386 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 370/717] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Jul 7 15:15:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:52.207047 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db65830-a816-4c7d-8bff-759eeed1a3c4 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:52.284255 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45893e5d-1ca8-4506-a8c8-d53370706206 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:52.359299 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db65830-a816-4c7d-8bff-759eeed1a3c4 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:52.388260 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7eecd0-044d-4a04-a8f8-cef63a076b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:52.388933 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7eecd0-044d-4a04-a8f8-cef63a076b66 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:52.389211 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7eecd0-044d-4a04-a8f8-cef63a076b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:52.389585 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7eecd0-044d-4a04-a8f8-cef63a076b66 None admin] PATH_INFO: `/v3/services` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:52.390221 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7eecd0-044d-4a04-a8f8-cef63a076b66 None admin] RBAC: Adding query filter params (type=network) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:52.390527 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7eecd0-044d-4a04-a8f8-cef63a076b66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:52.391945 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7eecd0-044d-4a04-a8f8-cef63a076b66 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:52.396038 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609cf08-d7fb-440f-9e63-54da785b3f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:52.397160 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609cf08-d7fb-440f-9e63-54da785b3f5d None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:52.397657 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609cf08-d7fb-440f-9e63-54da785b3f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:52.398134 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609cf08-d7fb-440f-9e63-54da785b3f5d None admin] PATH_INFO: `/v3/services` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:52.399109 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3609cf08-d7fb-440f-9e63-54da785b3f5d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:52.399644 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3609cf08-d7fb-440f-9e63-54da785b3f5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:52.401194 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3609cf08-d7fb-440f-9e63-54da785b3f5d None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:52.411594 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:52.411784 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 348/718] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Jul 7 15:15:52 2022] GET /identity/v3/services?type=network => generated 124 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:52.412321 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 371/719] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Jul 7 15:15:52 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:15:52.681067 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2dd55e-30af-4bc9-8618-26975bb0ba9f None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:52.681248 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2dd55e-30af-4bc9-8618-26975bb0ba9f None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:52.681535 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2dd55e-30af-4bc9-8618-26975bb0ba9f None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:52.683181 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 349/720] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:52.689055 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71353327-0f3c-4ced-a02d-ab99dbe87ff2 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:52.689351 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71353327-0f3c-4ced-a02d-ab99dbe87ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:52.689623 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71353327-0f3c-4ced-a02d-ab99dbe87ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:52.722498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-71353327-0f3c-4ced-a02d-ab99dbe87ff2 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:52.724011 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-71353327-0f3c-4ced-a02d-ab99dbe87ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:52.803937 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 372/721] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 15:15:52.826452 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2dd55e-30af-4bc9-8618-26975bb0ba9f None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:52.866103 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:52.943964 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2dd55e-30af-4bc9-8618-26975bb0ba9f None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:53.000209 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab88b12-be7d-42c6-9b4b-61ec3af1bed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:53.001002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab88b12-be7d-42c6-9b4b-61ec3af1bed3 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:53.001283 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab88b12-be7d-42c6-9b4b-61ec3af1bed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:53.001555 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab88b12-be7d-42c6-9b4b-61ec3af1bed3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:53.017281 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab88b12-be7d-42c6-9b4b-61ec3af1bed3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:53.025002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-6ab88b12-be7d-42c6-9b4b-61ec3af1bed3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:15:53.025002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:53.025002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:53.025002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:53.025002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:53.025002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:53.025002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:53.025002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:53.025002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:53.025002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:53.025002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:53.025002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:53.025002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:53.025002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:53.025002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:53.025002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:53.025951 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:53.025951 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:53.025951 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:53.025951 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:53.025951 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:53.025951 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:53.025951 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:53.025951 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:53.025951 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:53.025951 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:53.025951 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:53.025951 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:53.025951 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:53.025951 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:53.025951 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:53.025951 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:53.026792 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:53.026792 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:53.026792 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:53.026792 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:53.026792 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:53.026792 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:53.026792 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:53.026792 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:53.026792 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:53.026792 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:53.026792 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:15:53.026792 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:53.032177 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 350/722] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Jul 7 15:15:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:53.033426 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71353327-0f3c-4ced-a02d-ab99dbe87ff2 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:53.154242 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71353327-0f3c-4ced-a02d-ab99dbe87ff2 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:53.185238 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:53.193439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ebe61-9b13-47f0-8462-d4de2b2456ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:53.194171 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ebe61-9b13-47f0-8462-d4de2b2456ad None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:53.194404 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ebe61-9b13-47f0-8462-d4de2b2456ad None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:53.194683 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ebe61-9b13-47f0-8462-d4de2b2456ad None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:53.195443 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3ebe61-9b13-47f0-8462-d4de2b2456ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:53.195787 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3ebe61-9b13-47f0-8462-d4de2b2456ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:53.197165 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3ebe61-9b13-47f0-8462-d4de2b2456ad None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:53.211312 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 373/723] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:15:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:53.216875 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab88b12-be7d-42c6-9b4b-61ec3af1bed3 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:53.284545 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f8940-6043-4d91-8eb8-5888664cd1fe None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:53.284867 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f8940-6043-4d91-8eb8-5888664cd1fe None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:53.285116 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f8940-6043-4d91-8eb8-5888664cd1fe None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:53.287095 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 374/724] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:15:53.292395 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ee382-a251-44cf-b23a-d74ba5d82759 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:53.292687 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ee382-a251-44cf-b23a-d74ba5d82759 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:53.292968 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ee382-a251-44cf-b23a-d74ba5d82759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:53.313380 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-272ee382-a251-44cf-b23a-d74ba5d82759 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:53.314581 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-272ee382-a251-44cf-b23a-d74ba5d82759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:53.376076 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:53.382702 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab88b12-be7d-42c6-9b4b-61ec3af1bed3 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:53.438959 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ac3f9f-628d-4150-aae8-da95e500efca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:53.439785 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac3f9f-628d-4150-aae8-da95e500efca None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:53.440070 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac3f9f-628d-4150-aae8-da95e500efca None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:53.440344 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac3f9f-628d-4150-aae8-da95e500efca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:53.454356 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ac3f9f-628d-4150-aae8-da95e500efca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:53.455765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ac3f9f-628d-4150-aae8-da95e500efca None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:53.457491 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 375/725] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:53.485317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-e2ac3f9f-628d-4150-aae8-da95e500efca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:53.485317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:53.485317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:53.485317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:53.485317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:53.485317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:53.485317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:53.485317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:53.485317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:53.485317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:53.485317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:53.485317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:53.485317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:53.485317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:53.485317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:53.486282 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:53.486282 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:53.486282 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:53.486282 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:53.486282 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:53.486282 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:53.486282 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:53.486282 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:53.486282 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:53.486282 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:53.486282 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:53.486282 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:53.486282 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:53.486282 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:53.486282 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:53.486282 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:53.486282 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:53.487199 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:53.487199 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:53.487199 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:53.487199 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:53.487199 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 351/726] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:53.493402 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ee382-a251-44cf-b23a-d74ba5d82759 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:53.501477 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ac3f9f-628d-4150-aae8-da95e500efca None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:53.645144 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ac3f9f-628d-4150-aae8-da95e500efca None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:53.674383 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ee382-a251-44cf-b23a-d74ba5d82759 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:53.698164 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:53.715142 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53ec0d-943c-40ba-a3d0-eab9bde32acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:53.715780 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53ec0d-943c-40ba-a3d0-eab9bde32acd None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:53.716705 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53ec0d-943c-40ba-a3d0-eab9bde32acd None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:53.718020 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53ec0d-943c-40ba-a3d0-eab9bde32acd None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:53.719357 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc53ec0d-943c-40ba-a3d0-eab9bde32acd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:53.720385 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc53ec0d-943c-40ba-a3d0-eab9bde32acd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:53.722325 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc53ec0d-943c-40ba-a3d0-eab9bde32acd None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:53.726914 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b934dea8-ee7c-4825-ac3d-d802effc183e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:53.727988 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934dea8-ee7c-4825-ac3d-d802effc183e None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:53.728413 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934dea8-ee7c-4825-ac3d-d802effc183e None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:53.728829 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934dea8-ee7c-4825-ac3d-d802effc183e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:53.746674 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 376/727] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:53.749818 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53ec0d-943c-40ba-a3d0-eab9bde32acd None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:53.751043 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b934dea8-ee7c-4825-ac3d-d802effc183e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:53.758940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-b934dea8-ee7c-4825-ac3d-d802effc183e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:15:53.758940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:53.758940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:53.758940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:53.758940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:53.758940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:53.758940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:53.758940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:53.758940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:53.758940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:53.758940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:53.758940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:53.758940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:53.758940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:53.758940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:53.758940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:53.760074 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:53.760074 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:53.760074 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:53.760074 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:53.760074 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:53.760074 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:53.760074 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:53.760074 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:53.760074 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:53.760074 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:53.760074 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:53.760074 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:53.760074 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:53.760074 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:53.760074 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:53.760074 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:53.761097 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:53.761097 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:53.761097 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:53.761097 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:53.761097 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:53.761097 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:53.761097 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:53.761097 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:53.761097 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:53.761097 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:53.761097 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:15:53.761097 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:53.772040 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 352/728] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Jul 7 15:15:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:53.778956 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b934dea8-ee7c-4825-ac3d-d802effc183e None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:53.799219 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:53.938352 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53ec0d-943c-40ba-a3d0-eab9bde32acd None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:53.994710 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9330bcf0-eb86-4666-8f11-ea874546214b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:53.995841 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330bcf0-eb86-4666-8f11-ea874546214b None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:53.996265 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330bcf0-eb86-4666-8f11-ea874546214b None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:53.996861 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330bcf0-eb86-4666-8f11-ea874546214b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:54.011820 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9330bcf0-eb86-4666-8f11-ea874546214b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:54.014080 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b934dea8-ee7c-4825-ac3d-d802effc183e None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:54.017703 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9330bcf0-eb86-4666-8f11-ea874546214b None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:54.050610 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-9330bcf0-eb86-4666-8f11-ea874546214b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 07 15:15:54.050610 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:54.050610 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:54.050610 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:54.050610 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:54.050610 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:54.050610 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:54.050610 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:54.050610 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:54.050610 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:54.050610 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:54.050610 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:54.050610 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 15:15:54.050610 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 15:15:54.050610 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 15:15:54.050610 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 15:15:54.052220 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:54.052220 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:54.052220 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 15:15:54.052220 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:15:54.052220 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 15:15:54.052220 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:15:54.052220 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:54.052220 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:54.052220 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:54.052220 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:54.052220 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:54.052220 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:54.052220 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:54.052220 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:54.052220 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:54.052220 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:54.052220 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:54.052220 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:54.053362 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:54.053362 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:54.053362 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 15:15:54.053362 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 15:15:54.053362 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 15:15:54.053362 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 15:15:54.053362 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 15:15:54.053362 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 15:15:54.053362 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 07 15:15:54.053362 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:54.054517 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c5c1d9-8732-4be8-8875-285b68a3e408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:54.055285 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c5c1d9-8732-4be8-8875-285b68a3e408 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:54.055694 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c5c1d9-8732-4be8-8875-285b68a3e408 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:54.056144 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c5c1d9-8732-4be8-8875-285b68a3e408 None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:54.059616 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c5c1d9-8732-4be8-8875-285b68a3e408 None admin] RBAC: Adding query filter params (name=service) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:54.059616 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c5c1d9-8732-4be8-8875-285b68a3e408 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:54.059616 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c5c1d9-8732-4be8-8875-285b68a3e408 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:54.059908 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 377/729] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Jul 7 15:15:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:54.064989 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9330bcf0-eb86-4666-8f11-ea874546214b None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:54.075592 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 353/730] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Jul 7 15:15:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:54.077980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c5c1d9-8732-4be8-8875-285b68a3e408 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:54.138709 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:54.175157 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:54.196682 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9330bcf0-eb86-4666-8f11-ea874546214b None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:54.208029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c5c1d9-8732-4be8-8875-285b68a3e408 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:54.233968 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a7b34-4aae-4ef0-9f56-7f27b6c0d082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:54.234720 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a7b34-4aae-4ef0-9f56-7f27b6c0d082 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:54.235106 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a7b34-4aae-4ef0-9f56-7f27b6c0d082 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:54.235479 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a7b34-4aae-4ef0-9f56-7f27b6c0d082 None admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:54.236440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898a7b34-4aae-4ef0-9f56-7f27b6c0d082 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:54.236844 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898a7b34-4aae-4ef0-9f56-7f27b6c0d082 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:54.238378 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898a7b34-4aae-4ef0-9f56-7f27b6c0d082 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:54.245286 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6795d62-e229-4adb-b8dc-6ad7dd7404b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:54.246157 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6795d62-e229-4adb-b8dc-6ad7dd7404b2 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:54.246439 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6795d62-e229-4adb-b8dc-6ad7dd7404b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:54.246708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6795d62-e229-4adb-b8dc-6ad7dd7404b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:54.259327 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6795d62-e229-4adb-b8dc-6ad7dd7404b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:54.260791 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6795d62-e229-4adb-b8dc-6ad7dd7404b2 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:54.261784 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 378/731] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Jul 7 15:15:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:54.263249 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acb3be-4f12-4a6e-a468-ec35cf9117c2 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:54.263543 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acb3be-4f12-4a6e-a468-ec35cf9117c2 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:54.263845 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acb3be-4f12-4a6e-a468-ec35cf9117c2 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:54.265897 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 379/732] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:54.266598 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4acb3be-4f12-4a6e-a468-ec35cf9117c2 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:54.281293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-b6795d62-e229-4adb-b8dc-6ad7dd7404b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:54.281293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:54.281293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:54.281293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:54.281293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:54.281293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:54.281293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:54.281293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:54.281293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:54.281293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:54.281293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:54.281293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:54.281293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:54.281293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:54.281293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:54.282210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:54.282210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:54.282210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:54.282210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:54.282210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:54.282210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:54.282210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:54.282210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:54.282210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:54.282210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:54.282210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:54.282210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:54.282210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:54.282210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:54.282210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:54.282210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:54.282210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:54.283980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:54.283980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:54.283980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:54.283980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:54.283980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 354/733] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:54.284611 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd9576-c044-461d-b838-df1d28ee9a51 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:54.284752 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd9576-c044-461d-b838-df1d28ee9a51 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:54.286700 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd9576-c044-461d-b838-df1d28ee9a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:54.303853 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-64fd9576-c044-461d-b838-df1d28ee9a51 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:54.305111 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-64fd9576-c044-461d-b838-df1d28ee9a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:54.371452 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:54.375894 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4acb3be-4f12-4a6e-a468-ec35cf9117c2 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:54.383173 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 355/734] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:54.384237 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fd9576-c044-461d-b838-df1d28ee9a51 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:54.431384 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd2519b-3e1b-4ec7-b91c-b2d031a2b16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:54.432065 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2519b-3e1b-4ec7-b91c-b2d031a2b16e None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:54.432349 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2519b-3e1b-4ec7-b91c-b2d031a2b16e None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:54.432626 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2519b-3e1b-4ec7-b91c-b2d031a2b16e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:54.444625 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:54.498179 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fd9576-c044-461d-b838-df1d28ee9a51 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:54.506789 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd2519b-3e1b-4ec7-b91c-b2d031a2b16e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:54.508206 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd2519b-3e1b-4ec7-b91c-b2d031a2b16e None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:54.540107 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149843f5-c72e-456f-b52c-a39587be52d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:54.541078 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149843f5-c72e-456f-b52c-a39587be52d5 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:54.541507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149843f5-c72e-456f-b52c-a39587be52d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:54.541933 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149843f5-c72e-456f-b52c-a39587be52d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:54.543003 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149843f5-c72e-456f-b52c-a39587be52d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:54.543457 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149843f5-c72e-456f-b52c-a39587be52d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:54.545025 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149843f5-c72e-456f-b52c-a39587be52d5 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:54.558563 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 356/735] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:54.559744 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149843f5-c72e-456f-b52c-a39587be52d5 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:54.607478 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:54.615558 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 380/736] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:15:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 15:15:54.616494 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd2519b-3e1b-4ec7-b91c-b2d031a2b16e None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:54.673438 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149843f5-c72e-456f-b52c-a39587be52d5 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:54.688362 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:54.711930 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feea308-5519-4c2f-9c2b-33e16cecdd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:54.713044 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feea308-5519-4c2f-9c2b-33e16cecdd47 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:54.713562 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feea308-5519-4c2f-9c2b-33e16cecdd47 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:54.714017 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feea308-5519-4c2f-9c2b-33e16cecdd47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:54.723768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd2519b-3e1b-4ec7-b91c-b2d031a2b16e None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:54.727233 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feea308-5519-4c2f-9c2b-33e16cecdd47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:54.737171 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-2feea308-5519-4c2f-9c2b-33e16cecdd47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:15:54.737171 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:54.737171 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:54.737171 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:54.737171 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:54.737171 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:54.737171 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:54.737171 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:54.737171 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:54.737171 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:54.737171 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:54.737171 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:54.737171 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:54.737171 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:54.737171 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:54.737171 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:54.738161 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:54.738161 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:54.738161 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:54.738161 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:54.738161 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:54.738161 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:54.738161 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:54.738161 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:54.738161 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:54.738161 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:54.738161 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:54.738161 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:54.738161 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:54.738161 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:54.738161 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:54.738161 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:54.739049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:54.739049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:54.739049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:54.739049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:54.739049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:54.739049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:54.739049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:54.739049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:54.739049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:54.739049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:54.739049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:15:54.739049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:54.742043 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 357/737] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Jul 7 15:15:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:54.743095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feea308-5519-4c2f-9c2b-33e16cecdd47 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:54.765610 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4666b-a4c9-41d1-9059-46bfff516bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:54.766357 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4666b-a4c9-41d1-9059-46bfff516bea None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:54.766640 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4666b-a4c9-41d1-9059-46bfff516bea None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:54.766916 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4666b-a4c9-41d1-9059-46bfff516bea None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:54.774816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4666b-a4c9-41d1-9059-46bfff516bea None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:54.776302 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4666b-a4c9-41d1-9059-46bfff516bea None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:54.791218 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-e6c4666b-a4c9-41d1-9059-46bfff516bea None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 07 15:15:54.791218 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:54.791218 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:54.791218 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:54.791218 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:54.791218 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:54.791218 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:54.791218 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:54.791218 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:54.791218 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:54.791218 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:54.791218 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:54.791218 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 15:15:54.791218 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 15:15:54.791218 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 15:15:54.791218 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 15:15:54.792235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:54.792235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:54.792235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 15:15:54.792235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:15:54.792235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 15:15:54.792235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:15:54.792235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:54.792235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:54.792235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:54.792235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:54.792235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:54.792235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:54.792235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:54.792235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:54.792235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:54.792235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:54.792235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:54.792235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:54.793229 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:54.793229 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:54.793229 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 15:15:54.793229 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 15:15:54.793229 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 15:15:54.793229 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 15:15:54.793229 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 15:15:54.793229 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 15:15:54.793229 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 07 15:15:54.793229 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:54.793229 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 381/738] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Jul 7 15:15:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:54.793836 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4666b-a4c9-41d1-9059-46bfff516bea None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:54.855437 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:54.865331 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feea308-5519-4c2f-9c2b-33e16cecdd47 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:54.898463 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4666b-a4c9-41d1-9059-46bfff516bea None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:54.901330 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b6d6f8-84fc-4cce-a0fe-ea668d5acb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:54.902118 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6d6f8-84fc-4cce-a0fe-ea668d5acb1d None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:54.902397 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6d6f8-84fc-4cce-a0fe-ea668d5acb1d None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:54.902671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6d6f8-84fc-4cce-a0fe-ea668d5acb1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:54.912855 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b6d6f8-84fc-4cce-a0fe-ea668d5acb1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:54.914280 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b6d6f8-84fc-4cce-a0fe-ea668d5acb1d None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:54.932743 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-93b6d6f8-84fc-4cce-a0fe-ea668d5acb1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:54.932743 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:54.932743 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:54.932743 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:54.932743 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:54.932743 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:54.932743 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:54.932743 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:54.932743 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:54.932743 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:54.932743 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:54.932743 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:54.932743 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:54.932743 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:54.932743 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:54.940523 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:54.940523 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:54.940523 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:54.940523 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:54.940523 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:54.940523 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:54.940523 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:54.940523 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:54.940523 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:54.940523 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:54.940523 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:54.940523 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:54.940523 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:54.940523 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:54.940523 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:54.940523 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:54.940523 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:54.941621 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:54.941621 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:54.941621 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:54.941621 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:54.941621 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 358/739] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:54.941621 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b6d6f8-84fc-4cce-a0fe-ea668d5acb1d None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:54.943588 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21010391-8e13-431b-a846-13adb4ecdc47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:54.944274 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21010391-8e13-431b-a846-13adb4ecdc47 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:54.944599 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21010391-8e13-431b-a846-13adb4ecdc47 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:54.944879 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21010391-8e13-431b-a846-13adb4ecdc47 None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:54.945626 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21010391-8e13-431b-a846-13adb4ecdc47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:54.945934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21010391-8e13-431b-a846-13adb4ecdc47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:54.947310 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21010391-8e13-431b-a846-13adb4ecdc47 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:54.963788 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 382/740] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:15:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:54.963788 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a95b50f-3d3f-4049-abe4-82d65f6fce38 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:54.963788 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a95b50f-3d3f-4049-abe4-82d65f6fce38 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:54.963788 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a95b50f-3d3f-4049-abe4-82d65f6fce38 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:54.963788 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 383/741] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:54.963788 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a95b50f-3d3f-4049-abe4-82d65f6fce38 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:54.978604 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:55.052926 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b6d6f8-84fc-4cce-a0fe-ea668d5acb1d None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:55.069974 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a95b50f-3d3f-4049-abe4-82d65f6fce38 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:55.086406 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee4c0e-591c-489b-a865-06f05a14bf73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:55.087068 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee4c0e-591c-489b-a865-06f05a14bf73 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:55.087347 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee4c0e-591c-489b-a865-06f05a14bf73 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:55.087645 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee4c0e-591c-489b-a865-06f05a14bf73 None admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:55.088474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee4c0e-591c-489b-a865-06f05a14bf73 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:55.088782 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee4c0e-591c-489b-a865-06f05a14bf73 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:55.090196 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee4c0e-591c-489b-a865-06f05a14bf73 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:55.091883 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:55.103797 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc87146-252e-4191-ae03-9187544febc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:55.104457 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc87146-252e-4191-ae03-9187544febc3 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:55.104742 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc87146-252e-4191-ae03-9187544febc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:55.105776 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc87146-252e-4191-ae03-9187544febc3 None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:55.105776 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc87146-252e-4191-ae03-9187544febc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:55.106093 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc87146-252e-4191-ae03-9187544febc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:55.107443 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc87146-252e-4191-ae03-9187544febc3 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:55.109158 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 359/742] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Jul 7 15:15:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:55.109970 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee4c0e-591c-489b-a865-06f05a14bf73 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:55.124817 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 384/743] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:55.124817 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6cf52-217c-4a91-a9fc-8d6edb10dc5c None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:55.124817 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6cf52-217c-4a91-a9fc-8d6edb10dc5c None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:55.124817 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6cf52-217c-4a91-a9fc-8d6edb10dc5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:55.140182 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-f4e6cf52-217c-4a91-a9fc-8d6edb10dc5c None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:55.141364 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-f4e6cf52-217c-4a91-a9fc-8d6edb10dc5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:55.197965 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:55.206198 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 385/744] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:55.207484 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e4da0-e6c3-4a85-87f0-13fc5565852c None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:55.207798 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e4da0-e6c3-4a85-87f0-13fc5565852c None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:55.209354 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e4da0-e6c3-4a85-87f0-13fc5565852c None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:55.209354 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 386/745] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:55.210013 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037e4da0-e6c3-4a85-87f0-13fc5565852c None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:55.221963 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee4c0e-591c-489b-a865-06f05a14bf73 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:55.256802 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8798c8ca-780c-4068-876f-d9a9fd2dd526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:55.257585 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798c8ca-780c-4068-876f-d9a9fd2dd526 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:55.257864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798c8ca-780c-4068-876f-d9a9fd2dd526 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:55.258138 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798c8ca-780c-4068-876f-d9a9fd2dd526 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:55.268076 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8798c8ca-780c-4068-876f-d9a9fd2dd526 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:55.269494 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8798c8ca-780c-4068-876f-d9a9fd2dd526 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:55.287241 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-8798c8ca-780c-4068-876f-d9a9fd2dd526 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:55.287241 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:55.287241 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:55.287241 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:55.287241 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:55.287241 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:55.287241 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:55.287241 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:55.287241 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:55.287241 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:55.287241 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:55.287241 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:55.287241 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:55.287241 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:55.287241 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:55.288393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:55.288393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:55.288393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:55.288393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:55.288393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:55.288393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:55.288393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:55.288393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:55.288393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:55.288393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:55.288393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:55.288393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:55.288393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:55.288393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:55.288393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:55.288393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:55.288393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:55.289305 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:55.289305 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:55.289305 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:55.289305 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:55.289305 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 360/746] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:55.289305 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8798c8ca-780c-4068-876f-d9a9fd2dd526 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:55.323069 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037e4da0-e6c3-4a85-87f0-13fc5565852c None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:55.335264 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:55.356571 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a50c7d7-e6e7-4454-9464-71984c91bfe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:55.357252 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50c7d7-e6e7-4454-9464-71984c91bfe3 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:55.357539 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50c7d7-e6e7-4454-9464-71984c91bfe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:55.357815 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50c7d7-e6e7-4454-9464-71984c91bfe3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:55.387048 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8798c8ca-780c-4068-876f-d9a9fd2dd526 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:55.420395 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc01079-e4a3-4b39-9f8a-01b6f12b707d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:55.421739 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc01079-e4a3-4b39-9f8a-01b6f12b707d None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:55.422027 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc01079-e4a3-4b39-9f8a-01b6f12b707d None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:55.422296 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc01079-e4a3-4b39-9f8a-01b6f12b707d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:55.424174 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:55.424289 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a50c7d7-e6e7-4454-9464-71984c91bfe3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:55.425723 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a50c7d7-e6e7-4454-9464-71984c91bfe3 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:55.429559 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc01079-e4a3-4b39-9f8a-01b6f12b707d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:55.430971 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc01079-e4a3-4b39-9f8a-01b6f12b707d None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:55.444763 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-fdc01079-e4a3-4b39-9f8a-01b6f12b707d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:15:55.444763 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:55.444763 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:55.444763 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:55.444763 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:55.444763 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:55.444763 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:55.444763 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:55.444763 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:55.444763 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:55.444763 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:55.444763 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:55.444763 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 15:15:55.444763 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 15:15:55.444763 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 15:15:55.445648 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 15:15:55.445648 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:55.445648 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:55.445648 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:55.445648 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:55.445648 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:55.445648 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:55.445648 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:55.445648 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:55.445648 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:55.445648 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:55.445648 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:55.445648 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:55.445648 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:55.445648 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:55.445648 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:55.445648 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:55.446579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 15:15:55.446579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 15:15:55.446579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 15:15:55.446579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 15:15:55.446579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 15:15:55.446579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 15:15:55.446579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:15:55.446579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:55.446579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 361/747] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:15:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:55.447183 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8215f-974f-4e41-a8ff-bc645d988c63 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:55.447409 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8215f-974f-4e41-a8ff-bc645d988c63 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:55.447732 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8215f-974f-4e41-a8ff-bc645d988c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:55.464308 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-fbb8215f-974f-4e41-a8ff-bc645d988c63 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:55.465467 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-fbb8215f-974f-4e41-a8ff-bc645d988c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:55.532522 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 387/748] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:15:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 15:15:55.532522 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a50c7d7-e6e7-4454-9464-71984c91bfe3 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:55.532785 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 362/749] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 15:15:55.532785 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb8215f-974f-4e41-a8ff-bc645d988c63 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:55.551446 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:55.636043 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb8215f-974f-4e41-a8ff-bc645d988c63 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:55.636598 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a50c7d7-e6e7-4454-9464-71984c91bfe3 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:55.660807 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:55.671009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28370db-302f-47d7-ac7b-1537cd5d5733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:55.671491 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc9e646-4608-4a9a-883a-b2b5ccad4290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:55.671717 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28370db-302f-47d7-ac7b-1537cd5d5733 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:55.671952 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28370db-302f-47d7-ac7b-1537cd5d5733 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:55.672221 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28370db-302f-47d7-ac7b-1537cd5d5733 None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:55.672462 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc9e646-4608-4a9a-883a-b2b5ccad4290 None admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:55.672808 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc9e646-4608-4a9a-883a-b2b5ccad4290 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:55.672931 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28370db-302f-47d7-ac7b-1537cd5d5733 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:55.673204 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc9e646-4608-4a9a-883a-b2b5ccad4290 None admin] PATH_INFO: `/v3/services` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:55.673294 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28370db-302f-47d7-ac7b-1537cd5d5733 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:55.674028 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc9e646-4608-4a9a-883a-b2b5ccad4290 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:55.674569 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28370db-302f-47d7-ac7b-1537cd5d5733 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:55.675494 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc9e646-4608-4a9a-883a-b2b5ccad4290 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:55.685556 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 363/750] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:55.686364 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28370db-302f-47d7-ac7b-1537cd5d5733 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:55.787596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 388/751] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Jul 7 15:15:55 2022] POST /identity/v3/services => generated 251 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:15:55.787596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc9e646-4608-4a9a-883a-b2b5ccad4290 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:55.787914 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28370db-302f-47d7-ac7b-1537cd5d5733 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:55.840660 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adefdd6-7759-4185-b676-3bfa46673670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:55.841443 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adefdd6-7759-4185-b676-3bfa46673670 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:55.841730 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adefdd6-7759-4185-b676-3bfa46673670 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:55.842006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adefdd6-7759-4185-b676-3bfa46673670 None admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:55.843395 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adefdd6-7759-4185-b676-3bfa46673670 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:55.843737 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adefdd6-7759-4185-b676-3bfa46673670 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:55.845423 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adefdd6-7759-4185-b676-3bfa46673670 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:55.857071 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:55.863919 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 364/752] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Jul 7 15:15:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:55.865122 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adefdd6-7759-4185-b676-3bfa46673670 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:55.891705 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc9e646-4608-4a9a-883a-b2b5ccad4290 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:55.915014 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:55.932812 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e6b48-dd89-440f-9e66-a17be03bf0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:55.933610 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e6b48-dd89-440f-9e66-a17be03bf0ee None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:55.934499 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e6b48-dd89-440f-9e66-a17be03bf0ee None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:55.934499 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e6b48-dd89-440f-9e66-a17be03bf0ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:55.946428 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e6b48-dd89-440f-9e66-a17be03bf0ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:55.947950 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e6b48-dd89-440f-9e66-a17be03bf0ee None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:55.969394 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-cc8e6b48-dd89-440f-9e66-a17be03bf0ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:55.969394 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:55.969394 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:55.969394 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:55.969394 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:55.969394 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:55.969394 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:55.969394 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:55.969394 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:55.969394 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:55.969394 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:55.969394 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:55.969394 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:55.969394 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:55.969394 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:55.970292 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:55.970292 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:55.970292 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:55.970292 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:55.970292 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:55.970292 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:55.970292 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:55.970292 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:55.970292 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:55.970292 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:55.970292 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:55.970292 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:55.970292 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:55.970292 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:55.970292 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:55.970292 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:55.970292 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:55.971624 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:55.971624 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:55.971624 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:55.971624 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:55.971624 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 389/753] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:55.971624 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e6b48-dd89-440f-9e66-a17be03bf0ee None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:55.985527 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adefdd6-7759-4185-b676-3bfa46673670 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:56.020744 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f77221-e9d0-40c7-9136-605b6a7147b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:56.021445 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f77221-e9d0-40c7-9136-605b6a7147b2 None admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:56.021714 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f77221-e9d0-40c7-9136-605b6a7147b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:56.021963 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f77221-e9d0-40c7-9136-605b6a7147b2 None admin] PATH_INFO: `/v3/services` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:56.022712 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f77221-e9d0-40c7-9136-605b6a7147b2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:56.024151 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f77221-e9d0-40c7-9136-605b6a7147b2 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:56.079088 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e6b48-dd89-440f-9e66-a17be03bf0ee None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:56.101275 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:56.111412 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf2c503-c337-4769-abaf-8522ebc6a9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:56.112227 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2c503-c337-4769-abaf-8522ebc6a9eb None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:56.112509 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2c503-c337-4769-abaf-8522ebc6a9eb None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:56.112788 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2c503-c337-4769-abaf-8522ebc6a9eb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:56.118739 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 365/754] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Jul 7 15:15:55 2022] POST /identity/v3/services => generated 240 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:15:56.120597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf2c503-c337-4769-abaf-8522ebc6a9eb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:56.122053 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf2c503-c337-4769-abaf-8522ebc6a9eb None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:56.126898 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f77221-e9d0-40c7-9136-605b6a7147b2 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:56.137743 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-cbf2c503-c337-4769-abaf-8522ebc6a9eb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 07 15:15:56.137743 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:56.137743 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:56.137743 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:56.137743 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:56.137743 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:56.137743 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:56.137743 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:56.137743 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:56.137743 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:56.137743 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:56.137743 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:56.137743 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 15:15:56.137743 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 15:15:56.137743 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 15:15:56.137743 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 15:15:56.138810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:56.138810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:56.138810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 15:15:56.138810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:15:56.138810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 15:15:56.138810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:15:56.138810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:56.138810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:56.138810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:56.138810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:56.138810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:56.138810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:56.138810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:56.138810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:56.138810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:56.138810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:56.138810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:56.138810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:56.140374 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:56.140374 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:56.140374 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 15:15:56.140374 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 15:15:56.140374 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 15:15:56.140374 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 15:15:56.140374 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 15:15:56.140374 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 15:15:56.140374 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 07 15:15:56.140374 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:56.149559 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 390/755] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Jul 7 15:15:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:56.149559 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf2c503-c337-4769-abaf-8522ebc6a9eb None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:56.272571 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:56.277260 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf2c503-c337-4769-abaf-8522ebc6a9eb None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:56.282249 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f77221-e9d0-40c7-9136-605b6a7147b2 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:56.319456 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a7e4bf-ca7e-4d73-9a42-1ef19ef3be0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:56.320404 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7e4bf-ca7e-4d73-9a42-1ef19ef3be0e None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:56.320822 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7e4bf-ca7e-4d73-9a42-1ef19ef3be0e None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:56.321361 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7e4bf-ca7e-4d73-9a42-1ef19ef3be0e None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:56.322234 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a7e4bf-ca7e-4d73-9a42-1ef19ef3be0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:56.322688 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a7e4bf-ca7e-4d73-9a42-1ef19ef3be0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:56.324094 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a7e4bf-ca7e-4d73-9a42-1ef19ef3be0e None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:56.331008 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f590d7a-d1b7-4c6b-897f-aca14401cc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:56.332308 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f590d7a-d1b7-4c6b-897f-aca14401cc2d None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:56.332308 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f590d7a-d1b7-4c6b-897f-aca14401cc2d None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:56.332308 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f590d7a-d1b7-4c6b-897f-aca14401cc2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:56.333098 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f590d7a-d1b7-4c6b-897f-aca14401cc2d None admin] RBAC: Adding query filter params (role.id=0e7793fbf3064037984da651879e3a94, scope.project.id=c5260c7fd3614e9e93b167c86a94260e, user.id=b18033f849b240999d12c1885d2f7b91) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:56.333364 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f590d7a-d1b7-4c6b-897f-aca14401cc2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:56.334791 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f590d7a-d1b7-4c6b-897f-aca14401cc2d None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:56.338599 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 391/756] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:56.339653 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a7e4bf-ca7e-4d73-9a42-1ef19ef3be0e None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:56.342418 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 366/757] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:15:56 2022] GET /identity/v3/role_assignments?user.id=b18033f849b240999d12c1885d2f7b91&scope.project.id=c5260c7fd3614e9e93b167c86a94260e&role.id=0e7793fbf3064037984da651879e3a94 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:56.349037 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f590d7a-d1b7-4c6b-897f-aca14401cc2d None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:56.459331 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:56.513647 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f590d7a-d1b7-4c6b-897f-aca14401cc2d None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:56.559379 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12a0ba-816f-489d-9468-6263349da5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:56.560207 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12a0ba-816f-489d-9468-6263349da5fe None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:56.560485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12a0ba-816f-489d-9468-6263349da5fe None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:56.560759 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12a0ba-816f-489d-9468-6263349da5fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:56.563042 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a7e4bf-ca7e-4d73-9a42-1ef19ef3be0e None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:56.569001 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec12a0ba-816f-489d-9468-6263349da5fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:56.570562 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec12a0ba-816f-489d-9468-6263349da5fe None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:56.594489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-ec12a0ba-816f-489d-9468-6263349da5fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:15:56.594489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:56.594489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:56.594489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:56.594489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:56.594489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:56.594489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:56.594489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:56.594489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:56.594489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:56.594489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:56.594489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:56.594489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 15:15:56.594489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 15:15:56.594489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 15:15:56.595446 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 15:15:56.595446 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:56.595446 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:56.595446 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:56.595446 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:56.595446 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:56.595446 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:56.595446 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:56.595446 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:56.595446 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:56.595446 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:56.595446 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:56.595446 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:56.595446 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:56.595446 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:56.595446 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:56.595446 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:56.596411 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 15:15:56.596411 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 15:15:56.596411 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 15:15:56.596411 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 15:15:56.596411 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 15:15:56.596411 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 15:15:56.596411 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:15:56.596411 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:56.596411 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 367/758] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:15:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:56.602111 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12a0ba-816f-489d-9468-6263349da5fe None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:56.609741 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878ff4b-538c-46eb-86b9-0bd52b31967b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:56.610658 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878ff4b-538c-46eb-86b9-0bd52b31967b None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:56.611122 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878ff4b-538c-46eb-86b9-0bd52b31967b None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:56.611594 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878ff4b-538c-46eb-86b9-0bd52b31967b None admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:56.612638 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878ff4b-538c-46eb-86b9-0bd52b31967b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:56.613124 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878ff4b-538c-46eb-86b9-0bd52b31967b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:56.614735 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878ff4b-538c-46eb-86b9-0bd52b31967b None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:56.633014 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 392/759] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Jul 7 15:15:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:56.643585 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:56.643748 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878ff4b-538c-46eb-86b9-0bd52b31967b None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:56.753383 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12a0ba-816f-489d-9468-6263349da5fe None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:56.859912 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878ff4b-538c-46eb-86b9-0bd52b31967b None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:56.861725 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:56.916029 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c13bf-ef27-4755-a99a-f2c57b8ce4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:56.919883 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c13bf-ef27-4755-a99a-f2c57b8ce4ec None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:56.920471 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c13bf-ef27-4755-a99a-f2c57b8ce4ec None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:56.921462 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c13bf-ef27-4755-a99a-f2c57b8ce4ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:56.928685 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c951a14-49df-4c24-aaa0-493113aa6fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:56.929333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c951a14-49df-4c24-aaa0-493113aa6fde None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:56.930532 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c951a14-49df-4c24-aaa0-493113aa6fde None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:56.930844 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c951a14-49df-4c24-aaa0-493113aa6fde None admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:56.931659 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c951a14-49df-4c24-aaa0-493113aa6fde None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:56.931967 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c951a14-49df-4c24-aaa0-493113aa6fde None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:56.933388 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c951a14-49df-4c24-aaa0-493113aa6fde None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:56.947423 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9c13bf-ef27-4755-a99a-f2c57b8ce4ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:56.948922 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9c13bf-ef27-4755-a99a-f2c57b8ce4ec None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:56.961872 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 368/760] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Jul 7 15:15:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:15:56.973608 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c951a14-49df-4c24-aaa0-493113aa6fde None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:56.981995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-9b9c13bf-ef27-4755-a99a-f2c57b8ce4ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:56.981995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:56.981995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:56.981995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:56.981995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:56.981995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:56.981995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:56.981995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:56.981995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:56.981995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:56.981995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:56.981995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:56.981995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:15:56.981995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:15:56.981995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:15:56.983015 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:15:56.983015 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:56.983015 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:56.983015 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:56.983015 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:56.983015 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:56.983015 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:56.983015 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:56.983015 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:56.983015 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:56.983015 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:56.983015 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:56.983015 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:56.983015 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:56.983015 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:56.983015 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:56.983015 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:56.983960 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:15:56.983960 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:15:56.983960 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:15:56.983960 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:56.984186 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 393/761] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:56.989930 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c13bf-ef27-4755-a99a-f2c57b8ce4ec None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:57.101682 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c13bf-ef27-4755-a99a-f2c57b8ce4ec None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:57.141084 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e2e16-f7d0-432d-aca1-05a5e2218beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:57.141749 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e2e16-f7d0-432d-aca1-05a5e2218beb None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:57.142031 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e2e16-f7d0-432d-aca1-05a5e2218beb None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:57.142308 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e2e16-f7d0-432d-aca1-05a5e2218beb None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:57.143034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e2e16-f7d0-432d-aca1-05a5e2218beb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:57.143341 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e2e16-f7d0-432d-aca1-05a5e2218beb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:57.144742 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e2e16-f7d0-432d-aca1-05a5e2218beb None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:57.154485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:57.158408 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 394/762] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:15:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:15:57.163626 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e2e16-f7d0-432d-aca1-05a5e2218beb None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:57.195966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:57.209484 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c951a14-49df-4c24-aaa0-493113aa6fde None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:57.273927 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e2e16-f7d0-432d-aca1-05a5e2218beb None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:57.310059 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a59ced-9e23-4466-916a-57dad1532827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:57.310849 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a59ced-9e23-4466-916a-57dad1532827 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:57.311160 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a59ced-9e23-4466-916a-57dad1532827 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:57.311433 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a59ced-9e23-4466-916a-57dad1532827 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:57.318392 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a59ced-9e23-4466-916a-57dad1532827 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:57.319866 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a59ced-9e23-4466-916a-57dad1532827 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:57.332797 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9a957-8422-44af-9a9c-c6e0a1054986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:57.333452 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9a957-8422-44af-9a9c-c6e0a1054986 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:57.333731 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9a957-8422-44af-9a9c-c6e0a1054986 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:57.333999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9a957-8422-44af-9a9c-c6e0a1054986 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:57.334745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-63a59ced-9e23-4466-916a-57dad1532827 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:15:57.334745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:57.334745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:57.334745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:57.334745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:57.334745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:57.334745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:57.334745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:57.334745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:57.334745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:57.334745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:57.334745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:57.334745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 15:15:57.334745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 15:15:57.334745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 15:15:57.336554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 15:15:57.336554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:57.336554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:57.336554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:57.336554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:57.336554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:57.336554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:57.336554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:57.336554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:57.336554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:57.336554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:57.336554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:57.336554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:57.336554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:57.336554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:57.336554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:57.336554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:57.338109 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 15:15:57.338109 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 15:15:57.338109 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 15:15:57.338109 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 15:15:57.338109 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 15:15:57.338109 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 15:15:57.338109 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:15:57.338109 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:57.339087 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff9a957-8422-44af-9a9c-c6e0a1054986 None admin] RBAC: Adding query filter params (role.id=3d4f4e0a472e4ff9ac68d90c8612e956, scope.project.id=c5260c7fd3614e9e93b167c86a94260e, user.id=cd3c04f39d774998b66e6ce920925691) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:57.339087 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff9a957-8422-44af-9a9c-c6e0a1054986 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:57.339368 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 395/763] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:15:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:15:57.343585 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff9a957-8422-44af-9a9c-c6e0a1054986 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:57.346334 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a59ced-9e23-4466-916a-57dad1532827 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:57.355807 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 369/764] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:15:56 2022] GET /identity/v3/role_assignments?user.id=cd3c04f39d774998b66e6ce920925691&scope.project.id=c5260c7fd3614e9e93b167c86a94260e&role.id=3d4f4e0a472e4ff9ac68d90c8612e956 => generated 259 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:15:57.527876 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:57.555842 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a59ced-9e23-4466-916a-57dad1532827 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:57.602010 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca682f-d7f6-4dc4-b641-a8f2164da090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:57.603023 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca682f-d7f6-4dc4-b641-a8f2164da090 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:57.603509 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca682f-d7f6-4dc4-b641-a8f2164da090 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:57.604024 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca682f-d7f6-4dc4-b641-a8f2164da090 None admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:57.605033 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca682f-d7f6-4dc4-b641-a8f2164da090 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:57.605566 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca682f-d7f6-4dc4-b641-a8f2164da090 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:57.607295 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca682f-d7f6-4dc4-b641-a8f2164da090 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:57.619959 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 396/765] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Jul 7 15:15:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:57.627606 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9a957-8422-44af-9a9c-c6e0a1054986 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:57.793020 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9a957-8422-44af-9a9c-c6e0a1054986 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:57.845605 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5697338-ff2e-4e52-a588-041f49d7a91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:57.846490 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5697338-ff2e-4e52-a588-041f49d7a91c None admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:57.846766 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5697338-ff2e-4e52-a588-041f49d7a91c None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:57.847035 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5697338-ff2e-4e52-a588-041f49d7a91c None admin] PATH_INFO: `/v3/projects/c5260c7fd3614e9e93b167c86a94260e/users/de1632555cb345f2b9213da1925c02f4/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:57.849306 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5697338-ff2e-4e52-a588-041f49d7a91c None admin] RBAC: Authorizing `identity:create_grant(project_id=c5260c7fd3614e9e93b167c86a94260e, user_id=de1632555cb345f2b9213da1925c02f4, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:57.850715 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5697338-ff2e-4e52-a588-041f49d7a91c None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:57.967619 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 370/766] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Jul 7 15:15:57 2022] PUT /identity/v3/projects/c5260c7fd3614e9e93b167c86a94260e/users/de1632555cb345f2b9213da1925c02f4/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:15:58.030403 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8f645-27d3-4631-a667-fbc1dda46e59 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:58.030971 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8f645-27d3-4631-a667-fbc1dda46e59 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:58.031452 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8f645-27d3-4631-a667-fbc1dda46e59 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:58.033240 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 397/767] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:58.038780 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89fdc2-86f2-44b6-bf07-d8de8e9f44fa None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:58.039329 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89fdc2-86f2-44b6-bf07-d8de8e9f44fa None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:58.039807 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89fdc2-86f2-44b6-bf07-d8de8e9f44fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:58.047147 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:58.066625 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-9c89fdc2-86f2-44b6-bf07-d8de8e9f44fa None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:58.068150 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-9c89fdc2-86f2-44b6-bf07-d8de8e9f44fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:58.156587 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 371/768] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:15:58.201973 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe8f645-27d3-4631-a667-fbc1dda46e59 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:58.355773 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe8f645-27d3-4631-a667-fbc1dda46e59 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:58.394773 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76ea87c-66a9-4018-97d7-8353d2c62155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:58.395625 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ea87c-66a9-4018-97d7-8353d2c62155 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:58.395918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ea87c-66a9-4018-97d7-8353d2c62155 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:58.396198 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ea87c-66a9-4018-97d7-8353d2c62155 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:58.396867 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76ea87c-66a9-4018-97d7-8353d2c62155 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:58.398259 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76ea87c-66a9-4018-97d7-8353d2c62155 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:58.400198 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:58.417235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-a76ea87c-66a9-4018-97d7-8353d2c62155 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 07 15:15:58.417235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:58.417235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 15:15:58.417235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 15:15:58.417235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 15:15:58.417235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 15:15:58.417235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 15:15:58.417235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:15:58.417235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 07 15:15:58.417235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:58.417235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 15:15:58.417235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:58.417235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:58.417235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:58.417235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:58.417235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:58.418208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:58.418208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:58.418208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:58.418208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:58.418208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:58.418208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:58.418208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:58.418208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 15:15:58.418208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 15:15:58.418208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 15:15:58.418208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 15:15:58.418208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:58.418208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:58.418208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:58.418208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:58.418208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:58.419057 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:58.419057 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:58.419057 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:58.419057 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:58.419057 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:58.419057 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:58.419057 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:58.419057 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:58.419057 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:58.419057 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:58.419057 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:58.419057 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 15:15:58.419057 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:15:58.419057 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 07 15:15:58.419057 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:58.419057 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 398/769] 10.222.0.25 () {56 vars in 1129 bytes} [Thu Jul 7 15:15:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:15:58.426301 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c89fdc2-86f2-44b6-bf07-d8de8e9f44fa None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:58.572955 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb23b6-e84b-4f2f-9c4c-2201854d2508 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:58.573632 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb23b6-e84b-4f2f-9c4c-2201854d2508 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:58.574207 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb23b6-e84b-4f2f-9c4c-2201854d2508 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:58.576142 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 399/770] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:15:58.580519 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c89fdc2-86f2-44b6-bf07-d8de8e9f44fa None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:58.582063 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e56be-0274-4312-96d3-a0fdda1a629a None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:58.582561 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e56be-0274-4312-96d3-a0fdda1a629a None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:58.583043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e56be-0274-4312-96d3-a0fdda1a629a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:58.602895 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-350e56be-0274-4312-96d3-a0fdda1a629a None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:58.604534 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-350e56be-0274-4312-96d3-a0fdda1a629a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:58.638474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef590c-7422-489c-9e16-51810c34a46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:58.641080 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef590c-7422-489c-9e16-51810c34a46a None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:58.641659 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef590c-7422-489c-9e16-51810c34a46a None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:58.642273 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef590c-7422-489c-9e16-51810c34a46a None admin] PATH_INFO: `/v3/services` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:58.643817 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef590c-7422-489c-9e16-51810c34a46a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:58.644529 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef590c-7422-489c-9e16-51810c34a46a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:58.647549 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef590c-7422-489c-9e16-51810c34a46a None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:58.650445 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:58.658584 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 372/771] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Jul 7 15:15:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:58.668541 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef590c-7422-489c-9e16-51810c34a46a None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:58.698994 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 400/772] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:15:58.736298 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e56be-0274-4312-96d3-a0fdda1a629a None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:58.812154 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef590c-7422-489c-9e16-51810c34a46a None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:58.864333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7984b20-a300-4519-b15c-d8f017f88983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:58.865019 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7984b20-a300-4519-b15c-d8f017f88983 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:58.865304 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7984b20-a300-4519-b15c-d8f017f88983 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:58.865579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7984b20-a300-4519-b15c-d8f017f88983 None admin] PATH_INFO: `/v3/services` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:58.866297 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7984b20-a300-4519-b15c-d8f017f88983 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:58.866606 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7984b20-a300-4519-b15c-d8f017f88983 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:58.868016 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7984b20-a300-4519-b15c-d8f017f88983 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:58.874405 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 373/773] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Jul 7 15:15:58 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:58.879921 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7984b20-a300-4519-b15c-d8f017f88983 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:58.895674 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e56be-0274-4312-96d3-a0fdda1a629a None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:58.942912 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:58.963031 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2dd1ea-84ef-4a6f-8d1c-c7ee4bdd72b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:58.963891 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2dd1ea-84ef-4a6f-8d1c-c7ee4bdd72b1 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:58.964171 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2dd1ea-84ef-4a6f-8d1c-c7ee4bdd72b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:58.964921 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2dd1ea-84ef-4a6f-8d1c-c7ee4bdd72b1 None admin] PATH_INFO: `/v3/services/network` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:58.965084 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2dd1ea-84ef-4a6f-8d1c-c7ee4bdd72b1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:58.966529 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2dd1ea-84ef-4a6f-8d1c-c7ee4bdd72b1 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:58.988348 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-ff2dd1ea-84ef-4a6f-8d1c-c7ee4bdd72b1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 07 15:15:58.988348 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:58.988348 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 15:15:58.988348 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 15:15:58.988348 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 15:15:58.988348 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 15:15:58.988348 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 15:15:58.988348 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:15:58.988348 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 07 15:15:58.988348 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:58.988348 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 15:15:58.988348 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:58.988348 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:58.988348 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:58.988348 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:58.988348 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:58.988348 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:58.989398 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:58.989398 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:58.989398 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:58.989398 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:58.989398 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:58.989398 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:58.989398 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 15:15:58.989398 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 15:15:58.989398 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 15:15:58.989398 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 15:15:58.989398 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:58.989398 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:58.989398 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:58.989398 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:58.989398 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:58.989398 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:58.990257 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:58.990257 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:58.990257 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:58.990257 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:58.990257 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:58.990257 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:58.990257 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:58.990257 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:58.990257 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:58.990257 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:58.990257 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 15:15:58.990257 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:15:58.990257 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 07 15:15:58.990257 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:15:58.993482 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 401/774] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:15:58 2022] GET /identity/v3/services/network => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:15:58.997352 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2dd1ea-84ef-4a6f-8d1c-c7ee4bdd72b1 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:59.001432 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:59.069762 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7984b20-a300-4519-b15c-d8f017f88983 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:59.116338 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc21670-4ae5-4a09-b692-7b09bf3fc0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:59.117550 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc21670-4ae5-4a09-b692-7b09bf3fc0a1 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:59.117842 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc21670-4ae5-4a09-b692-7b09bf3fc0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:59.118123 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc21670-4ae5-4a09-b692-7b09bf3fc0a1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:59.119206 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc21670-4ae5-4a09-b692-7b09bf3fc0a1 None admin] RBAC: Adding query filter params (interface=public, service_id=cee161fd842241d3b92d507c13a80285, region_id=RegionOne) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:59.119521 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc21670-4ae5-4a09-b692-7b09bf3fc0a1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:59.121833 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc21670-4ae5-4a09-b692-7b09bf3fc0a1 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:59.131872 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2dd1ea-84ef-4a6f-8d1c-c7ee4bdd72b1 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:59.132415 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 374/775] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Jul 7 15:15:58 2022] GET /identity/v3/endpoints?service_id=cee161fd842241d3b92d507c13a80285&interface=public®ion_id=RegionOne => generated 194 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:15:59.136147 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb265e-be83-46b8-aa05-40085a07587c None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:59.136434 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb265e-be83-46b8-aa05-40085a07587c None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:59.136712 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb265e-be83-46b8-aa05-40085a07587c None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:59.138318 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 375/776] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:59.139055 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beb265e-be83-46b8-aa05-40085a07587c None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:59.183920 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9e535c-3923-4d1e-b9e9-405b0466d44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:59.184528 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e535c-3923-4d1e-b9e9-405b0466d44d None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:59.184815 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e535c-3923-4d1e-b9e9-405b0466d44d None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:59.185093 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e535c-3923-4d1e-b9e9-405b0466d44d None admin] PATH_INFO: `/v3/services` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:59.185810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9e535c-3923-4d1e-b9e9-405b0466d44d None admin] RBAC: Adding query filter params (name=network) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:59.186701 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9e535c-3923-4d1e-b9e9-405b0466d44d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:59.188076 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9e535c-3923-4d1e-b9e9-405b0466d44d None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:59.193722 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 402/777] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Jul 7 15:15:58 2022] GET /identity/v3/services?name=network => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:15:59.194583 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7668f9-0f4f-4063-a979-7fe504924dd5 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:59.194876 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7668f9-0f4f-4063-a979-7fe504924dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:59.195158 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7668f9-0f4f-4063-a979-7fe504924dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:59.213362 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-2f7668f9-0f4f-4063-a979-7fe504924dd5 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:59.214647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-2f7668f9-0f4f-4063-a979-7fe504924dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:59.275746 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beb265e-be83-46b8-aa05-40085a07587c None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:59.290045 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:59.303274 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:59.312819 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b9ac9-f1b9-4c74-a4f2-69a653afb44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:59.313486 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b9ac9-f1b9-4c74-a4f2-69a653afb44e None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:59.313769 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b9ac9-f1b9-4c74-a4f2-69a653afb44e None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:59.314045 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b9ac9-f1b9-4c74-a4f2-69a653afb44e None admin] PATH_INFO: `/v3/services` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:59.314839 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037b9ac9-f1b9-4c74-a4f2-69a653afb44e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:59.321097 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037b9ac9-f1b9-4c74-a4f2-69a653afb44e None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:59.321246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 403/778] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:15:59.321246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7668f9-0f4f-4063-a979-7fe504924dd5 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:59.330091 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 376/779] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Jul 7 15:15:59 2022] GET /identity/v3/services => generated 772 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:59.370549 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b9ac9-f1b9-4c74-a4f2-69a653afb44e None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:59.510427 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7668f9-0f4f-4063-a979-7fe504924dd5 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:59.554814 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f24a1-1ac3-49b3-af2b-12d02c045865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:59.555492 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f24a1-1ac3-49b3-af2b-12d02c045865 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:59.556302 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f24a1-1ac3-49b3-af2b-12d02c045865 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:59.556588 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f24a1-1ac3-49b3-af2b-12d02c045865 None admin] PATH_INFO: `/v3/services` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:59.557613 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3f24a1-1ac3-49b3-af2b-12d02c045865 None admin] RBAC: Adding query filter params (type=network) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:59.557926 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3f24a1-1ac3-49b3-af2b-12d02c045865 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:59.559832 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3f24a1-1ac3-49b3-af2b-12d02c045865 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:59.569143 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 404/780] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Jul 7 15:15:59 2022] GET /identity/v3/services?type=network => generated 350 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:59.574721 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f24a1-1ac3-49b3-af2b-12d02c045865 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:59.577025 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b9ac9-f1b9-4c74-a4f2-69a653afb44e None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:59.618418 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164e439-3d61-4e9a-8e04-57ee61f07205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:59.619293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164e439-3d61-4e9a-8e04-57ee61f07205 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:59.619592 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164e439-3d61-4e9a-8e04-57ee61f07205 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:59.619885 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164e439-3d61-4e9a-8e04-57ee61f07205 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:59.633174 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3164e439-3d61-4e9a-8e04-57ee61f07205 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:15:59.642718 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-3164e439-3d61-4e9a-8e04-57ee61f07205 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:15:59.642718 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:15:59.642718 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:15:59.642718 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:15:59.642718 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:15:59.642718 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:15:59.642718 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:15:59.642718 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:15:59.642718 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:15:59.642718 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:15:59.642718 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:15:59.642718 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:15:59.642718 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:15:59.642718 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:15:59.642718 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:15:59.642718 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:15:59.643711 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:15:59.643711 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:15:59.643711 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:15:59.643711 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:15:59.643711 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:15:59.643711 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:15:59.643711 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:15:59.643711 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:15:59.643711 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:15:59.643711 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:15:59.643711 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:15:59.643711 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:15:59.643711 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:15:59.643711 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:15:59.643711 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:15:59.643711 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:15:59.644587 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:15:59.644587 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:15:59.644587 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:15:59.644587 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:15:59.644587 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:15:59.644587 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:15:59.644587 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:15:59.644587 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:15:59.644587 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:15:59.644587 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:15:59.644587 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:15:59.644587 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:15:59.645398 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 377/781] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Jul 7 15:15:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:15:59.646260 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:59.653523 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164e439-3d61-4e9a-8e04-57ee61f07205 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:59.728051 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f24a1-1ac3-49b3-af2b-12d02c045865 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:59.746615 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:59.769771 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c498e4a5-de35-4e35-bcc2-954c4fa60e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:59.770403 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c498e4a5-de35-4e35-bcc2-954c4fa60e3f None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:59.770690 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c498e4a5-de35-4e35-bcc2-954c4fa60e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:59.770965 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c498e4a5-de35-4e35-bcc2-954c4fa60e3f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:59.771795 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c498e4a5-de35-4e35-bcc2-954c4fa60e3f None admin] RBAC: Adding query filter params (interface=public, service_id=c263261c925e4646ba42efb1c48d5372, region_id=RegionOne) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:59.772151 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c498e4a5-de35-4e35-bcc2-954c4fa60e3f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:59.773678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c498e4a5-de35-4e35-bcc2-954c4fa60e3f None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:59.782020 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 405/782] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Jul 7 15:15:59 2022] GET /identity/v3/endpoints?service_id=c263261c925e4646ba42efb1c48d5372&interface=public®ion_id=RegionOne => generated 194 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:15:59.783516 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0168ac-e6b3-4c58-8c4d-01077bc4b6e7 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:59.783652 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0168ac-e6b3-4c58-8c4d-01077bc4b6e7 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:59.788000 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0168ac-e6b3-4c58-8c4d-01077bc4b6e7 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:59.789548 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 406/783] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:15:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:15:59.790991 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0168ac-e6b3-4c58-8c4d-01077bc4b6e7 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:59.808508 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164e439-3d61-4e9a-8e04-57ee61f07205 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:59.845571 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4879c00-3d3d-4519-86c5-723e41f44ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:59.846245 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4879c00-3d3d-4519-86c5-723e41f44ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:59.846529 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4879c00-3d3d-4519-86c5-723e41f44ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:59.846804 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4879c00-3d3d-4519-86c5-723e41f44ce2 None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:59.847556 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4879c00-3d3d-4519-86c5-723e41f44ce2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:15:59.848157 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4879c00-3d3d-4519-86c5-723e41f44ce2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:59.849519 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4879c00-3d3d-4519-86c5-723e41f44ce2 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:15:59.860919 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 378/784] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:15:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:15:59.862251 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62ea19-3a70-4d5c-b823-c4c8ffcde251 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:59.862532 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62ea19-3a70-4d5c-b823-c4c8ffcde251 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:59.862799 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62ea19-3a70-4d5c-b823-c4c8ffcde251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:59.888322 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-9f62ea19-3a70-4d5c-b823-c4c8ffcde251 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:15:59.889579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-9f62ea19-3a70-4d5c-b823-c4c8ffcde251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:15:59.944601 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0168ac-e6b3-4c58-8c4d-01077bc4b6e7 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:15:59.965765 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:15:59.989542 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 379/785] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:15:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:15:59.990544 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f62ea19-3a70-4d5c-b823-c4c8ffcde251 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:15:59.991742 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349d6a0-43c2-4ad3-a16a-324a57d679fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:15:59.992654 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349d6a0-43c2-4ad3-a16a-324a57d679fc None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:15:59.993429 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349d6a0-43c2-4ad3-a16a-324a57d679fc None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:15:59.993940 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349d6a0-43c2-4ad3-a16a-324a57d679fc None admin] PATH_INFO: `/v3/services` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:15:59.995756 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3349d6a0-43c2-4ad3-a16a-324a57d679fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:15:59.997356 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3349d6a0-43c2-4ad3-a16a-324a57d679fc None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:00.005092 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 407/786] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Jul 7 15:15:59 2022] GET /identity/v3/services => generated 772 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:00.020752 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349d6a0-43c2-4ad3-a16a-324a57d679fc None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:00.042776 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:00.128842 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f62ea19-3a70-4d5c-b823-c4c8ffcde251 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:00.192118 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85958619-c48d-4e56-ab25-4e6a0b46491b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:00.192939 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85958619-c48d-4e56-ab25-4e6a0b46491b None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:00.193226 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85958619-c48d-4e56-ab25-4e6a0b46491b None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:00.193507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85958619-c48d-4e56-ab25-4e6a0b46491b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:00.205971 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85958619-c48d-4e56-ab25-4e6a0b46491b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:00.208089 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85958619-c48d-4e56-ab25-4e6a0b46491b None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:00.210149 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349d6a0-43c2-4ad3-a16a-324a57d679fc None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:00.231643 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-85958619-c48d-4e56-ab25-4e6a0b46491b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:00.231643 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:00.231643 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:00.231643 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:00.231643 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:00.231643 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:00.231643 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:00.231643 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:00.231643 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:00.231643 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:00.231643 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:00.231643 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:00.231643 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:16:00.231643 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:16:00.231643 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:16:00.232661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:16:00.232661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:00.232661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:00.232661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:00.232661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:00.232661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:00.232661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:00.232661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:00.232661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:00.232661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:00.232661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:00.232661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:00.232661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:00.232661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:00.232661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:00.232661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:00.232661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:00.233652 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:16:00.233652 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:16:00.233652 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:00.233652 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:00.233652 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 380/787] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:15:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:00.241084 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85958619-c48d-4e56-ab25-4e6a0b46491b None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:00.266244 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84f759-be31-4bcf-8c2d-1cc6c02440f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:00.267048 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84f759-be31-4bcf-8c2d-1cc6c02440f6 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:00.267331 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84f759-be31-4bcf-8c2d-1cc6c02440f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:00.267628 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84f759-be31-4bcf-8c2d-1cc6c02440f6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:00.282142 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f84f759-be31-4bcf-8c2d-1cc6c02440f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:16:00.292059 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-2f84f759-be31-4bcf-8c2d-1cc6c02440f6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:16:00.292059 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:00.292059 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:00.292059 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:00.292059 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:00.292059 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:00.292059 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:00.292059 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:00.292059 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:00.292059 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:00.292059 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:00.292059 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:00.292059 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:16:00.292059 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:16:00.292059 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:16:00.292059 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:16:00.293054 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:16:00.293054 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:16:00.293054 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:16:00.293054 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:16:00.293054 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:00.293054 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:00.293054 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:00.293054 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:00.293054 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:00.293054 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:00.293054 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:00.293054 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:00.293054 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:00.293054 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:00.293054 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:00.293054 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:00.293924 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:00.293924 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:00.293924 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:00.293924 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:00.293924 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:16:00.293924 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:16:00.293924 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:16:00.293924 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:16:00.293924 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:16:00.293924 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:16:00.293924 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:16:00.293924 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:00.293924 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 408/788] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Jul 7 15:16:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:00.299140 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84f759-be31-4bcf-8c2d-1cc6c02440f6 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:00.346896 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:00.365789 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:00.379390 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85958619-c48d-4e56-ab25-4e6a0b46491b None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:00.411359 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84f759-be31-4bcf-8c2d-1cc6c02440f6 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:00.420932 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb1bfc0-6748-4fd6-bfaf-c712e70d9476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:00.421825 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1bfc0-6748-4fd6-bfaf-c712e70d9476 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:00.422287 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1bfc0-6748-4fd6-bfaf-c712e70d9476 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:00.422669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1bfc0-6748-4fd6-bfaf-c712e70d9476 None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:00.423804 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb1bfc0-6748-4fd6-bfaf-c712e70d9476 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:00.424030 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb1bfc0-6748-4fd6-bfaf-c712e70d9476 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:00.425565 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb1bfc0-6748-4fd6-bfaf-c712e70d9476 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:00.439117 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 381/789] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:16:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:00.452720 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb1bfc0-6748-4fd6-bfaf-c712e70d9476 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:00.455136 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c1fef2-b294-48ea-8319-bc2467554d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:00.456147 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1fef2-b294-48ea-8319-bc2467554d82 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:00.456580 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1fef2-b294-48ea-8319-bc2467554d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:00.457004 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1fef2-b294-48ea-8319-bc2467554d82 None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:00.458004 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c1fef2-b294-48ea-8319-bc2467554d82 None admin] RBAC: Adding query filter params (name=service) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:00.458523 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c1fef2-b294-48ea-8319-bc2467554d82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:00.460270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c1fef2-b294-48ea-8319-bc2467554d82 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:00.469457 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 409/790] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Jul 7 15:16:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:00.471049 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f241e-2c08-48d3-91ca-a863ab117ad8 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:00.471503 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f241e-2c08-48d3-91ca-a863ab117ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:00.471987 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f241e-2c08-48d3-91ca-a863ab117ad8 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:00.473997 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 410/791] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:16:00.479609 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f241e-2c08-48d3-91ca-a863ab117ad8 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:00.578953 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb1bfc0-6748-4fd6-bfaf-c712e70d9476 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:00.589040 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:00.593421 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f241e-2c08-48d3-91ca-a863ab117ad8 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:00.615051 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595eccc2-3206-4056-b8ce-abbaa00350cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:00.615865 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595eccc2-3206-4056-b8ce-abbaa00350cc None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:00.616142 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595eccc2-3206-4056-b8ce-abbaa00350cc None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:00.616421 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595eccc2-3206-4056-b8ce-abbaa00350cc None admin] PATH_INFO: `/v3/users/swift` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:00.626708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595eccc2-3206-4056-b8ce-abbaa00350cc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:00.628352 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595eccc2-3206-4056-b8ce-abbaa00350cc None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:00.633435 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c239bd-3f8d-4f4e-a0d3-554835df82bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:00.634447 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c239bd-3f8d-4f4e-a0d3-554835df82bb None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:00.634908 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c239bd-3f8d-4f4e-a0d3-554835df82bb None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:00.635389 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c239bd-3f8d-4f4e-a0d3-554835df82bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:00.643327 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-595eccc2-3206-4056-b8ce-abbaa00350cc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 07 15:16:00.643327 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:00.643327 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:00.643327 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:00.643327 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:00.643327 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:00.643327 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:00.643327 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:00.643327 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:00.643327 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:00.643327 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:00.643327 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:00.643327 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 15:16:00.643327 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 15:16:00.643327 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 15:16:00.643327 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 15:16:00.644311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:00.644311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:00.644311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 15:16:00.644311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:16:00.644311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 15:16:00.644311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:16:00.644311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:00.644311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:00.644311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:00.644311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:00.644311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:00.644311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:00.644311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:00.644311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:00.644311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:00.644311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:00.644311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:00.644311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:00.645270 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:00.645270 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:00.645270 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 15:16:00.645270 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 15:16:00.645270 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 15:16:00.645270 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 15:16:00.645270 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 15:16:00.645270 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 15:16:00.645270 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 07 15:16:00.645270 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:00.649198 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 382/792] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Jul 7 15:16:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:16:00.650356 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c239bd-3f8d-4f4e-a0d3-554835df82bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:00.650503 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62dc13b-55b5-4b94-933c-14d7e29a4d37 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:00.650706 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62dc13b-55b5-4b94-933c-14d7e29a4d37 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:00.650975 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62dc13b-55b5-4b94-933c-14d7e29a4d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:00.652667 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c239bd-3f8d-4f4e-a0d3-554835df82bb None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:00.668316 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-f62dc13b-55b5-4b94-933c-14d7e29a4d37 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:00.669481 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-f62dc13b-55b5-4b94-933c-14d7e29a4d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:00.672517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-c3c239bd-3f8d-4f4e-a0d3-554835df82bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:00.672517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:00.672517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:00.672517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:00.672517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:00.672517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:00.672517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:00.672517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:00.672517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:00.672517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:00.672517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:00.672517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:00.672517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:16:00.672517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:16:00.672517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:16:00.673415 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:16:00.673415 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:00.673415 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:00.673415 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:00.673415 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:00.673415 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:00.673415 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:00.673415 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:00.673415 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:00.673415 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:00.673415 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:00.673415 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:00.673415 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:00.673415 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:00.673415 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:00.673415 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:00.673415 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:00.674335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:16:00.674335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:16:00.674335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:00.674335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:00.676313 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 411/793] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:16:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:00.677301 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c239bd-3f8d-4f4e-a0d3-554835df82bb None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:00.683043 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:00.762108 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 383/794] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:16:00.762984 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62dc13b-55b5-4b94-933c-14d7e29a4d37 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:00.794905 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c239bd-3f8d-4f4e-a0d3-554835df82bb None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:00.834512 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d68529-f289-401b-9c65-b0178690ddca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:00.835642 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d68529-f289-401b-9c65-b0178690ddca None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:00.836075 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d68529-f289-401b-9c65-b0178690ddca None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:00.836497 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d68529-f289-401b-9c65-b0178690ddca None admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:00.837479 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d68529-f289-401b-9c65-b0178690ddca None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:00.837980 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d68529-f289-401b-9c65-b0178690ddca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:00.839757 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d68529-f289-401b-9c65-b0178690ddca None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:00.860914 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 412/795] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Jul 7 15:16:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:00.862170 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d68529-f289-401b-9c65-b0178690ddca None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:00.876214 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:00.894077 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62dc13b-55b5-4b94-933c-14d7e29a4d37 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:00.938192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93350780-abaa-4ee6-9902-029dbcf008ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:00.939123 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93350780-abaa-4ee6-9902-029dbcf008ef None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:00.939547 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93350780-abaa-4ee6-9902-029dbcf008ef None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:00.939988 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93350780-abaa-4ee6-9902-029dbcf008ef None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:00.940845 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93350780-abaa-4ee6-9902-029dbcf008ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:00.941307 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93350780-abaa-4ee6-9902-029dbcf008ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:00.942801 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93350780-abaa-4ee6-9902-029dbcf008ef None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:00.965405 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 384/796] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:16:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:00.966433 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93350780-abaa-4ee6-9902-029dbcf008ef None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:00.971419 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:00.985883 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d68529-f289-401b-9c65-b0178690ddca None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:01.032798 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69aa608-99f3-46ac-aa8e-e5bc0a888a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:01.034014 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69aa608-99f3-46ac-aa8e-e5bc0a888a5e None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:01.034303 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69aa608-99f3-46ac-aa8e-e5bc0a888a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:01.034844 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69aa608-99f3-46ac-aa8e-e5bc0a888a5e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:01.049566 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69aa608-99f3-46ac-aa8e-e5bc0a888a5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:16:01.058791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-d69aa608-99f3-46ac-aa8e-e5bc0a888a5e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:16:01.058791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:01.058791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:01.058791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:01.058791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:01.058791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:01.058791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:01.058791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:01.058791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:01.058791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:01.058791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:01.058791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:01.058791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:16:01.058791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:16:01.058791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:16:01.058791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:16:01.060035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:16:01.060035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:16:01.060035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:16:01.060035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:16:01.060035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:01.060035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:01.060035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:01.060035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:01.060035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:01.060035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:01.060035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:01.060035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:01.060035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:01.060035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:01.060035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:01.060035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:01.061498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:01.061498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:01.061498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:01.061498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:01.061498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:16:01.061498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:16:01.061498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:16:01.061498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:16:01.061498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:16:01.061498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:16:01.061498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:16:01.061498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:01.061498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 413/797] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Jul 7 15:16:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:01.062228 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69aa608-99f3-46ac-aa8e-e5bc0a888a5e None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:01.091588 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93350780-abaa-4ee6-9902-029dbcf008ef None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:01.129327 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd3079-80c7-4eae-ab3b-11d73c2b1dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:01.130210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd3079-80c7-4eae-ab3b-11d73c2b1dba None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:01.130527 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd3079-80c7-4eae-ab3b-11d73c2b1dba None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:01.130827 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd3079-80c7-4eae-ab3b-11d73c2b1dba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:01.142447 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd3079-80c7-4eae-ab3b-11d73c2b1dba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:01.143950 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd3079-80c7-4eae-ab3b-11d73c2b1dba None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:01.157676 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:01.163243 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-05bd3079-80c7-4eae-ab3b-11d73c2b1dba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:01.163243 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:01.163243 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:01.163243 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:01.163243 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:01.163243 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:01.163243 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:01.163243 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:01.163243 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:01.163243 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:01.163243 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:01.163243 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:01.163243 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:16:01.163243 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:16:01.163243 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:16:01.164379 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:16:01.164379 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:01.164379 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:01.164379 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:01.164379 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:01.164379 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:01.164379 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:01.164379 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:01.164379 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:01.164379 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:01.164379 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:01.164379 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:01.164379 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:01.164379 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:01.164379 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:01.164379 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:01.164379 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:01.165524 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:16:01.165524 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:16:01.165524 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:01.165524 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:01.179236 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 385/798] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:16:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:01.179236 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd3079-80c7-4eae-ab3b-11d73c2b1dba None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:01.191601 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69aa608-99f3-46ac-aa8e-e5bc0a888a5e None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:01.237629 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2dd66-e1fa-404c-95bf-776b889cf98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:01.238581 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2dd66-e1fa-404c-95bf-776b889cf98e None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:01.238974 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2dd66-e1fa-404c-95bf-776b889cf98e None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:01.239254 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2dd66-e1fa-404c-95bf-776b889cf98e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:01.243351 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:01.250632 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a2dd66-e1fa-404c-95bf-776b889cf98e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:01.252123 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a2dd66-e1fa-404c-95bf-776b889cf98e None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:01.271733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-e8a2dd66-e1fa-404c-95bf-776b889cf98e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 07 15:16:01.271733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:01.271733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:01.271733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:01.271733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:01.271733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:01.271733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:01.271733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:01.271733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:01.271733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:01.271733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:01.271733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:01.271733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 15:16:01.271733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 15:16:01.271733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 15:16:01.271733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 15:16:01.277785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:01.277785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:01.277785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 15:16:01.277785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:16:01.277785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 15:16:01.277785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:16:01.277785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:01.277785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:01.277785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:01.277785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:01.277785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:01.277785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:01.277785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:01.277785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:01.277785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:01.277785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:01.277785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:01.277785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:01.278833 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:01.278833 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:01.278833 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 15:16:01.278833 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 15:16:01.278833 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 15:16:01.278833 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 15:16:01.278833 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 15:16:01.278833 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 15:16:01.278833 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 07 15:16:01.278833 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:01.278833 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 414/799] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Jul 7 15:16:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:01.278833 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2dd66-e1fa-404c-95bf-776b889cf98e None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:01.304675 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd3079-80c7-4eae-ab3b-11d73c2b1dba None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:01.350309 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02428f23-09ad-4697-8fd1-971b7e0919b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:01.350923 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02428f23-09ad-4697-8fd1-971b7e0919b2 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:01.351791 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02428f23-09ad-4697-8fd1-971b7e0919b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:01.352081 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02428f23-09ad-4697-8fd1-971b7e0919b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:01.352839 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02428f23-09ad-4697-8fd1-971b7e0919b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:01.353146 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02428f23-09ad-4697-8fd1-971b7e0919b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:01.354800 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02428f23-09ad-4697-8fd1-971b7e0919b2 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:01.371023 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 386/800] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:16:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:01.371023 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02428f23-09ad-4697-8fd1-971b7e0919b2 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:01.398168 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2dd66-e1fa-404c-95bf-776b889cf98e None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:01.440244 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:01.448270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ae8e8-2c43-4a1a-917e-a3377d3eaa76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:01.449170 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ae8e8-2c43-4a1a-917e-a3377d3eaa76 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:01.449451 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ae8e8-2c43-4a1a-917e-a3377d3eaa76 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:01.449723 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ae8e8-2c43-4a1a-917e-a3377d3eaa76 None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:01.451341 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7ae8e8-2c43-4a1a-917e-a3377d3eaa76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:01.452213 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7ae8e8-2c43-4a1a-917e-a3377d3eaa76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:01.453640 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7ae8e8-2c43-4a1a-917e-a3377d3eaa76 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:01.469405 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 415/801] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:16:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:01.471408 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ae8e8-2c43-4a1a-917e-a3377d3eaa76 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:01.494521 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02428f23-09ad-4697-8fd1-971b7e0919b2 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:01.516670 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:01.539983 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d30b74-ca8e-4d44-bf2f-78f0fd776366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:01.540663 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d30b74-ca8e-4d44-bf2f-78f0fd776366 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:01.540945 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d30b74-ca8e-4d44-bf2f-78f0fd776366 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:01.541218 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d30b74-ca8e-4d44-bf2f-78f0fd776366 None admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:01.542058 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d30b74-ca8e-4d44-bf2f-78f0fd776366 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:01.542364 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d30b74-ca8e-4d44-bf2f-78f0fd776366 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:01.543832 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d30b74-ca8e-4d44-bf2f-78f0fd776366 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:01.568590 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 387/802] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Jul 7 15:16:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:01.570145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d30b74-ca8e-4d44-bf2f-78f0fd776366 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:01.598980 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ae8e8-2c43-4a1a-917e-a3377d3eaa76 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:01.645567 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a98b4e-db50-4c20-b2f0-3eb2162d9451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:01.646969 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a98b4e-db50-4c20-b2f0-3eb2162d9451 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:01.647452 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a98b4e-db50-4c20-b2f0-3eb2162d9451 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:01.648467 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a98b4e-db50-4c20-b2f0-3eb2162d9451 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:01.662665 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a98b4e-db50-4c20-b2f0-3eb2162d9451 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:01.665253 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a98b4e-db50-4c20-b2f0-3eb2162d9451 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:01.683058 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d30b74-ca8e-4d44-bf2f-78f0fd776366 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:01.684036 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-77a98b4e-db50-4c20-b2f0-3eb2162d9451 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:01.684036 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:01.684036 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:01.684036 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:01.684036 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:01.684036 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:01.684036 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:01.684036 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:01.684036 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:01.684036 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:01.684036 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:01.684036 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:01.684036 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:16:01.684036 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:16:01.684036 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:16:01.685108 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:16:01.685108 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:01.685108 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:01.685108 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:01.685108 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:01.685108 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:01.685108 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:01.685108 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:01.685108 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:01.685108 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:01.685108 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:01.685108 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:01.685108 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:01.685108 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:01.685108 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:01.685108 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:01.685108 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:01.686255 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:16:01.686255 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:16:01.686255 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:01.686255 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:01.687535 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 416/803] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:16:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:01.688270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a98b4e-db50-4c20-b2f0-3eb2162d9451 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:01.726605 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7201a761-1f2d-4285-b638-77b3b0426e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:01.727444 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201a761-1f2d-4285-b638-77b3b0426e99 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:01.727749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201a761-1f2d-4285-b638-77b3b0426e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:01.728074 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201a761-1f2d-4285-b638-77b3b0426e99 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:01.733471 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:01.735300 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7201a761-1f2d-4285-b638-77b3b0426e99 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:01.736813 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7201a761-1f2d-4285-b638-77b3b0426e99 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:01.750659 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-7201a761-1f2d-4285-b638-77b3b0426e99 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:16:01.750659 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:01.750659 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:01.750659 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:01.750659 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:01.750659 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:01.750659 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:01.750659 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:01.750659 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:01.750659 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:01.750659 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:01.750659 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:01.750659 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 15:16:01.750659 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 15:16:01.750659 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 15:16:01.751784 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 15:16:01.751784 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:01.751784 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:01.751784 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:01.751784 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:01.751784 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:01.751784 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:01.751784 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:01.751784 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:01.751784 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:01.751784 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:01.751784 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:01.751784 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:01.751784 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:01.751784 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:01.751784 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:01.751784 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:01.752922 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 15:16:01.752922 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 15:16:01.752922 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 15:16:01.752922 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 15:16:01.752922 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 15:16:01.752922 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 15:16:01.752922 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:16:01.752922 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:01.756625 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 388/804] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:16:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:01.757978 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0401566-1364-41ab-b7c0-8406fbc7a68e None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:01.758264 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0401566-1364-41ab-b7c0-8406fbc7a68e None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:01.758540 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0401566-1364-41ab-b7c0-8406fbc7a68e None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:01.760178 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 389/805] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:16:01.760872 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0401566-1364-41ab-b7c0-8406fbc7a68e None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:01.797534 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a98b4e-db50-4c20-b2f0-3eb2162d9451 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:01.799763 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:01.837650 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa32ff-69d5-40ad-b249-8458cbff2ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:01.838631 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa32ff-69d5-40ad-b249-8458cbff2ada None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:01.839002 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa32ff-69d5-40ad-b249-8458cbff2ada None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:01.839371 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa32ff-69d5-40ad-b249-8458cbff2ada None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:01.851027 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fa32ff-69d5-40ad-b249-8458cbff2ada None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:01.853014 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fa32ff-69d5-40ad-b249-8458cbff2ada None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:01.870578 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-85fa32ff-69d5-40ad-b249-8458cbff2ada None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:01.870578 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:01.870578 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:01.870578 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:01.870578 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:01.870578 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:01.870578 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:01.870578 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:01.870578 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:01.870578 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:01.870578 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:01.870578 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:01.870578 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:16:01.870578 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:16:01.870578 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:16:01.874848 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:16:01.874848 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:01.874848 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:01.874848 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:01.874848 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:01.874848 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:01.874848 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:01.874848 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:01.874848 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:01.874848 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:01.874848 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:01.874848 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:01.874848 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:01.874848 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:01.874848 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:01.874848 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:01.874848 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:01.876061 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:16:01.876061 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:16:01.876061 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:01.876061 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:01.876362 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0401566-1364-41ab-b7c0-8406fbc7a68e None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:01.876450 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 417/806] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:16:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:01.876450 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa32ff-69d5-40ad-b249-8458cbff2ada None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:01.912493 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8cb3c9-f9a1-46af-92e2-43fb47b4b8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:01.913111 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8cb3c9-f9a1-46af-92e2-43fb47b4b8ce None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:01.913396 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8cb3c9-f9a1-46af-92e2-43fb47b4b8ce None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:01.913687 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8cb3c9-f9a1-46af-92e2-43fb47b4b8ce None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:01.914394 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8cb3c9-f9a1-46af-92e2-43fb47b4b8ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:01.914695 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8cb3c9-f9a1-46af-92e2-43fb47b4b8ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:01.916119 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8cb3c9-f9a1-46af-92e2-43fb47b4b8ce None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:01.935014 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 390/807] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:16:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:01.935014 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d7990-7668-4d14-94d8-83e035932e53 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:01.935014 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d7990-7668-4d14-94d8-83e035932e53 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:01.935014 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d7990-7668-4d14-94d8-83e035932e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:01.950897 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-716d7990-7668-4d14-94d8-83e035932e53 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:01.952724 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-716d7990-7668-4d14-94d8-83e035932e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:01.988249 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:01.995978 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa32ff-69d5-40ad-b249-8458cbff2ada None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:02.027084 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 391/808] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:16:02.027972 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716d7990-7668-4d14-94d8-83e035932e53 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:02.042128 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b61b053-65ba-48b3-a15c-1792dbc20199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:02.043439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61b053-65ba-48b3-a15c-1792dbc20199 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:02.044167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61b053-65ba-48b3-a15c-1792dbc20199 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:02.044729 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61b053-65ba-48b3-a15c-1792dbc20199 None admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:02.046185 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b61b053-65ba-48b3-a15c-1792dbc20199 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:02.046686 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b61b053-65ba-48b3-a15c-1792dbc20199 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:02.048331 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b61b053-65ba-48b3-a15c-1792dbc20199 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:02.055185 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:02.062449 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 418/809] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Jul 7 15:16:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:02.067393 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b61b053-65ba-48b3-a15c-1792dbc20199 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:02.131292 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716d7990-7668-4d14-94d8-83e035932e53 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:02.165093 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe71166-a253-4476-8ebf-9a73bbe2187a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:02.165672 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe71166-a253-4476-8ebf-9a73bbe2187a None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:02.165950 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe71166-a253-4476-8ebf-9a73bbe2187a None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:02.166224 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe71166-a253-4476-8ebf-9a73bbe2187a None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:02.166939 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe71166-a253-4476-8ebf-9a73bbe2187a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:02.167239 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe71166-a253-4476-8ebf-9a73bbe2187a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:02.168616 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe71166-a253-4476-8ebf-9a73bbe2187a None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:02.172327 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b61b053-65ba-48b3-a15c-1792dbc20199 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:02.180189 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 392/810] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:16:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:02.180978 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe71166-a253-4476-8ebf-9a73bbe2187a None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:02.212509 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e86db69-0f88-4b2f-a227-267de6b0c021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:02.213483 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e86db69-0f88-4b2f-a227-267de6b0c021 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:02.214077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e86db69-0f88-4b2f-a227-267de6b0c021 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:02.214353 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e86db69-0f88-4b2f-a227-267de6b0c021 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:02.222129 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e86db69-0f88-4b2f-a227-267de6b0c021 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:02.224046 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e86db69-0f88-4b2f-a227-267de6b0c021 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:02.238647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-8e86db69-0f88-4b2f-a227-267de6b0c021 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 07 15:16:02.238647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:02.238647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:02.238647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:02.238647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:02.238647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:02.238647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:02.238647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:02.238647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:02.238647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:02.238647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:02.238647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:02.238647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 15:16:02.238647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 15:16:02.238647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 15:16:02.238647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 15:16:02.239862 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:02.239862 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:02.239862 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 15:16:02.239862 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:16:02.239862 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 15:16:02.239862 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:16:02.239862 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:02.239862 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:02.239862 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:02.239862 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:02.239862 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:02.239862 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:02.239862 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:02.239862 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:02.239862 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:02.239862 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:02.239862 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:02.239862 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:02.240860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:02.240860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:02.240860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 15:16:02.240860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 15:16:02.240860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 15:16:02.240860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 15:16:02.240860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 15:16:02.240860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 15:16:02.240860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 07 15:16:02.240860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:02.240860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 419/811] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Jul 7 15:16:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:02.240860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e86db69-0f88-4b2f-a227-267de6b0c021 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:02.257046 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:02.285499 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:02.301981 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe71166-a253-4476-8ebf-9a73bbe2187a None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:02.340113 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d2a70-7410-414d-a3a3-660b265a8134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:02.340901 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d2a70-7410-414d-a3a3-660b265a8134 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:02.341178 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d2a70-7410-414d-a3a3-660b265a8134 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:02.341452 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d2a70-7410-414d-a3a3-660b265a8134 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:02.342125 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d2a70-7410-414d-a3a3-660b265a8134 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:02.343159 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e86db69-0f88-4b2f-a227-267de6b0c021 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:02.343706 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d2a70-7410-414d-a3a3-660b265a8134 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:02.357351 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-a09d2a70-7410-414d-a3a3-660b265a8134 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 07 15:16:02.357351 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:02.357351 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 15:16:02.357351 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 15:16:02.357351 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 15:16:02.357351 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 15:16:02.357351 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 15:16:02.357351 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:16:02.357351 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 07 15:16:02.357351 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:02.357351 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 15:16:02.357351 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:02.357351 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:02.357351 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:02.357351 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:02.357351 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:02.358908 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:02.358908 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:02.358908 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:02.358908 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:02.358908 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:02.358908 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:02.358908 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:02.358908 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 15:16:02.358908 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 15:16:02.358908 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 15:16:02.358908 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 15:16:02.358908 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:02.358908 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:02.358908 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:02.358908 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:02.358908 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:02.365070 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:02.365070 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:02.365070 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:02.365070 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:02.365070 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:02.365070 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:02.365070 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:02.365070 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:02.365070 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:02.365070 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:02.365070 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:02.365070 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 15:16:02.365070 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:16:02.365070 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 07 15:16:02.365070 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:02.365070 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 393/812] 10.222.0.25 () {56 vars in 1129 bytes} [Thu Jul 7 15:16:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:02.366028 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d2a70-7410-414d-a3a3-660b265a8134 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:02.378917 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1624c6-52d0-4893-9867-36cb0dc3eff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:02.379903 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1624c6-52d0-4893-9867-36cb0dc3eff6 None admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:02.380184 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1624c6-52d0-4893-9867-36cb0dc3eff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:02.380463 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1624c6-52d0-4893-9867-36cb0dc3eff6 None admin] PATH_INFO: `/v3/projects/c5260c7fd3614e9e93b167c86a94260e/users/b18033f849b240999d12c1885d2f7b91/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:02.382400 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1624c6-52d0-4893-9867-36cb0dc3eff6 None admin] RBAC: Authorizing `identity:create_grant(project_id=c5260c7fd3614e9e93b167c86a94260e, user_id=b18033f849b240999d12c1885d2f7b91, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:02.383857 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1624c6-52d0-4893-9867-36cb0dc3eff6 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:02.459203 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d2a70-7410-414d-a3a3-660b265a8134 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:02.474714 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 420/813] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Jul 7 15:16:02 2022] PUT /identity/v3/projects/c5260c7fd3614e9e93b167c86a94260e/users/b18033f849b240999d12c1885d2f7b91/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:16:02.475504 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1624c6-52d0-4893-9867-36cb0dc3eff6 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:02.506924 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb528be3-ce18-4fb1-8efc-2b4d1d17f848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:02.507825 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb528be3-ce18-4fb1-8efc-2b4d1d17f848 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:02.508109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb528be3-ce18-4fb1-8efc-2b4d1d17f848 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:02.508653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb528be3-ce18-4fb1-8efc-2b4d1d17f848 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:02.517203 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb528be3-ce18-4fb1-8efc-2b4d1d17f848 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:02.518663 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb528be3-ce18-4fb1-8efc-2b4d1d17f848 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:02.534211 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-cb528be3-ce18-4fb1-8efc-2b4d1d17f848 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:16:02.534211 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:02.534211 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:02.534211 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:02.534211 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:02.534211 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:02.534211 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:02.534211 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:02.534211 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:02.534211 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:02.534211 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:02.534211 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:02.534211 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 15:16:02.534211 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 15:16:02.534211 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 15:16:02.535410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 15:16:02.535410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:02.535410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:02.535410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:02.535410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:02.535410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:02.535410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:02.535410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:02.535410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:02.535410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:02.535410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:02.535410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:02.535410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:02.535410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:02.535410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:02.535410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:02.535410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:02.536577 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 15:16:02.536577 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 15:16:02.536577 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 15:16:02.536577 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 15:16:02.536577 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 15:16:02.536577 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 15:16:02.536577 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:16:02.536577 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:02.536577 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 394/814] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:16:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:02.537191 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befcaf6d-a21b-4343-a910-8b6ee7b73263 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:02.537353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befcaf6d-a21b-4343-a910-8b6ee7b73263 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:02.537629 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befcaf6d-a21b-4343-a910-8b6ee7b73263 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:02.539045 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 395/815] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:16:02.539577 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befcaf6d-a21b-4343-a910-8b6ee7b73263 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:02.553395 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:02.572448 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:02.601572 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1624c6-52d0-4893-9867-36cb0dc3eff6 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:02.641126 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e35a07-32f7-45b7-b631-62e4a553914a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:02.642086 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e35a07-32f7-45b7-b631-62e4a553914a None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:02.642904 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e35a07-32f7-45b7-b631-62e4a553914a None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:02.643182 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e35a07-32f7-45b7-b631-62e4a553914a None admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:02.644067 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e35a07-32f7-45b7-b631-62e4a553914a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:02.644376 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e35a07-32f7-45b7-b631-62e4a553914a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:02.645814 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e35a07-32f7-45b7-b631-62e4a553914a None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:02.661017 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befcaf6d-a21b-4343-a910-8b6ee7b73263 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:02.665354 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 421/816] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Jul 7 15:16:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:02.666176 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e35a07-32f7-45b7-b631-62e4a553914a None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:02.703834 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd6d37d-0937-4dfa-a1e8-62f27bb6de8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:02.704501 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd6d37d-0937-4dfa-a1e8-62f27bb6de8e None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:02.704783 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd6d37d-0937-4dfa-a1e8-62f27bb6de8e None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:02.705056 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd6d37d-0937-4dfa-a1e8-62f27bb6de8e None admin] PATH_INFO: `/v3/services` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:02.705763 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd6d37d-0937-4dfa-a1e8-62f27bb6de8e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:02.706070 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd6d37d-0937-4dfa-a1e8-62f27bb6de8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:02.707873 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd6d37d-0937-4dfa-a1e8-62f27bb6de8e None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:02.716018 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 396/817] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Jul 7 15:16:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:02.716018 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fdd4b-7a03-4a05-974e-7c32637e28e1 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:02.716018 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fdd4b-7a03-4a05-974e-7c32637e28e1 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:02.716018 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fdd4b-7a03-4a05-974e-7c32637e28e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:02.735093 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-fb7fdd4b-7a03-4a05-974e-7c32637e28e1 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:02.736316 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-fb7fdd4b-7a03-4a05-974e-7c32637e28e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:02.772705 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e35a07-32f7-45b7-b631-62e4a553914a None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:02.804412 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:02.809621 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788f726-c003-4fe0-a7f1-18edaee827c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:02.810285 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788f726-c003-4fe0-a7f1-18edaee827c4 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:02.810572 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788f726-c003-4fe0-a7f1-18edaee827c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:02.810882 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788f726-c003-4fe0-a7f1-18edaee827c4 None admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:02.811711 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788f726-c003-4fe0-a7f1-18edaee827c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:02.812025 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788f726-c003-4fe0-a7f1-18edaee827c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:02.815823 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 397/818] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:16:02.815823 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7fdd4b-7a03-4a05-974e-7c32637e28e1 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:02.816010 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788f726-c003-4fe0-a7f1-18edaee827c4 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:02.820731 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:02.831542 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 422/819] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Jul 7 15:16:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:02.832462 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788f726-c003-4fe0-a7f1-18edaee827c4 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:02.932069 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7fdd4b-7a03-4a05-974e-7c32637e28e1 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:02.939726 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788f726-c003-4fe0-a7f1-18edaee827c4 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:02.975621 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a6750-05c4-4771-9ece-55356e5d55a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:02.976340 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a6750-05c4-4771-9ece-55356e5d55a0 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:02.976621 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a6750-05c4-4771-9ece-55356e5d55a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:02.977094 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a6750-05c4-4771-9ece-55356e5d55a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:02.983042 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa979a-1988-4600-8fc0-ea36da8e545b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:02.983853 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa979a-1988-4600-8fc0-ea36da8e545b None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:02.984203 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa979a-1988-4600-8fc0-ea36da8e545b None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:02.984556 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa979a-1988-4600-8fc0-ea36da8e545b None admin] PATH_INFO: `/v3/services` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:02.985331 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fa979a-1988-4600-8fc0-ea36da8e545b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:02.986056 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fa979a-1988-4600-8fc0-ea36da8e545b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:02.987424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fa979a-1988-4600-8fc0-ea36da8e545b None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:02.993282 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 423/820] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Jul 7 15:16:02 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:02.998260 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa979a-1988-4600-8fc0-ea36da8e545b None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:03.046949 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848a6750-05c4-4771-9ece-55356e5d55a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:03.048425 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848a6750-05c4-4771-9ece-55356e5d55a0 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:03.051916 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:03.058891 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:03.103206 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa979a-1988-4600-8fc0-ea36da8e545b None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:03.138451 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479dbefe-8d0f-46a5-8d8f-f5a74aad6630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:03.139875 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479dbefe-8d0f-46a5-8d8f-f5a74aad6630 None admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:03.140149 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479dbefe-8d0f-46a5-8d8f-f5a74aad6630 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:03.140846 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479dbefe-8d0f-46a5-8d8f-f5a74aad6630 None admin] PATH_INFO: `/v3/projects/c5260c7fd3614e9e93b167c86a94260e/users/cd3c04f39d774998b66e6ce920925691/roles/3d4f4e0a472e4ff9ac68d90c8612e956` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:03.158847 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 398/821] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:16:02 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 07 15:16:03.158847 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a6750-05c4-4771-9ece-55356e5d55a0 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:03.160579 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479dbefe-8d0f-46a5-8d8f-f5a74aad6630 None admin] RBAC: Authorizing `identity:create_grant(project_id=c5260c7fd3614e9e93b167c86a94260e, user_id=cd3c04f39d774998b66e6ce920925691, role_id=3d4f4e0a472e4ff9ac68d90c8612e956)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:03.162067 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479dbefe-8d0f-46a5-8d8f-f5a74aad6630 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:03.231154 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 424/822] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Jul 7 15:16:02 2022] PUT /identity/v3/projects/c5260c7fd3614e9e93b167c86a94260e/users/cd3c04f39d774998b66e6ce920925691/roles/3d4f4e0a472e4ff9ac68d90c8612e956 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:16:03.236980 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479dbefe-8d0f-46a5-8d8f-f5a74aad6630 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:03.278103 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a6750-05c4-4771-9ece-55356e5d55a0 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:03.281698 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:03.332410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa77fb3-4f3c-4e9e-97c7-75b1b8b9440e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:03.333317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa77fb3-4f3c-4e9e-97c7-75b1b8b9440e None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:03.333665 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa77fb3-4f3c-4e9e-97c7-75b1b8b9440e None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:03.334003 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa77fb3-4f3c-4e9e-97c7-75b1b8b9440e None admin] PATH_INFO: `/v3/services/network` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:03.334809 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa77fb3-4f3c-4e9e-97c7-75b1b8b9440e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:03.337056 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa77fb3-4f3c-4e9e-97c7-75b1b8b9440e None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:03.354419 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-0fa77fb3-4f3c-4e9e-97c7-75b1b8b9440e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 07 15:16:03.354419 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:03.354419 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 15:16:03.354419 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 15:16:03.354419 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 15:16:03.354419 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 15:16:03.354419 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 15:16:03.354419 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:16:03.354419 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 07 15:16:03.354419 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:03.354419 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 15:16:03.354419 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:03.354419 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:03.354419 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:03.354419 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:03.354419 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:03.354419 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:03.355499 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:03.355499 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:03.355499 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:03.355499 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:03.355499 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:03.355499 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:03.355499 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 15:16:03.355499 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 15:16:03.355499 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 15:16:03.355499 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 15:16:03.355499 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:03.355499 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:03.355499 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:03.355499 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:03.355499 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:03.355499 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:03.356397 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:03.356397 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:03.356397 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:03.356397 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:03.356397 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:03.356397 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:03.356397 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:03.356397 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:03.356397 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:03.356397 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:03.356397 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 15:16:03.356397 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:16:03.356397 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 07 15:16:03.356397 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:03.359725 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 399/823] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:16:03 2022] GET /identity/v3/services/network => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:03.360873 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa77fb3-4f3c-4e9e-97c7-75b1b8b9440e None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:03.371787 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479dbefe-8d0f-46a5-8d8f-f5a74aad6630 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:03.382113 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:03.415906 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ba343-6a73-49d7-9384-726081773647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:03.416713 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ba343-6a73-49d7-9384-726081773647 None admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:03.417111 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ba343-6a73-49d7-9384-726081773647 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:03.417483 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ba343-6a73-49d7-9384-726081773647 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:03.418318 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24ba343-6a73-49d7-9384-726081773647 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:03.420329 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24ba343-6a73-49d7-9384-726081773647 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:03.492137 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa77fb3-4f3c-4e9e-97c7-75b1b8b9440e None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:03.521461 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 425/824] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Jul 7 15:16:03 2022] POST /identity/v3/endpoints => generated 346 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:16:03.528174 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ba343-6a73-49d7-9384-726081773647 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:03.538786 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ef1e8-95f6-410d-98b6-4a493034de04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:03.539934 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ef1e8-95f6-410d-98b6-4a493034de04 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:03.540404 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ef1e8-95f6-410d-98b6-4a493034de04 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:03.540885 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ef1e8-95f6-410d-98b6-4a493034de04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:03.554661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351ef1e8-95f6-410d-98b6-4a493034de04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:03.556469 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351ef1e8-95f6-410d-98b6-4a493034de04 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:03.578708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-351ef1e8-95f6-410d-98b6-4a493034de04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:03.578708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:03.578708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:03.578708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:03.578708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:03.578708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:03.578708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:03.578708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:03.578708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:03.578708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:03.578708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:03.578708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:03.578708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:16:03.578708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:16:03.578708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:16:03.579734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:16:03.579734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:03.579734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:03.579734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:03.579734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:03.579734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:03.579734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:03.579734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:03.579734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:03.579734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:03.579734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:03.579734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:03.579734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:03.579734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:03.579734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:03.579734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:03.579734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:03.580656 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:16:03.580656 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:16:03.580656 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:03.580656 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:03.583738 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 400/825] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:16:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:03.591585 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ef1e8-95f6-410d-98b6-4a493034de04 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:03.669390 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:03.679486 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ba343-6a73-49d7-9384-726081773647 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:03.719600 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f4b57-cbe7-418f-abfc-a3cb8dcc4a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:03.720318 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f4b57-cbe7-418f-abfc-a3cb8dcc4a5b None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:03.720587 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f4b57-cbe7-418f-abfc-a3cb8dcc4a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:03.720867 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f4b57-cbe7-418f-abfc-a3cb8dcc4a5b None admin] PATH_INFO: `/v3/services` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:03.721587 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7f4b57-cbe7-418f-abfc-a3cb8dcc4a5b None admin] RBAC: Adding query filter params (name=network) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:03.721896 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7f4b57-cbe7-418f-abfc-a3cb8dcc4a5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:03.723553 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7f4b57-cbe7-418f-abfc-a3cb8dcc4a5b None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:03.728795 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 426/826] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Jul 7 15:16:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:03.737622 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f4b57-cbe7-418f-abfc-a3cb8dcc4a5b None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:03.768327 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:03.841855 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ef1e8-95f6-410d-98b6-4a493034de04 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:03.858569 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f4b57-cbe7-418f-abfc-a3cb8dcc4a5b None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:03.894799 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b5618a-9578-42ed-af13-2c65f3d4db3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:03.896485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5618a-9578-42ed-af13-2c65f3d4db3b None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:03.897135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5618a-9578-42ed-af13-2c65f3d4db3b None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:03.899437 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5618a-9578-42ed-af13-2c65f3d4db3b None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:03.900417 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b5618a-9578-42ed-af13-2c65f3d4db3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:03.901371 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b5618a-9578-42ed-af13-2c65f3d4db3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:03.903922 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b5618a-9578-42ed-af13-2c65f3d4db3b None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:03.917725 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81486a-2948-4b68-99cf-0b15d3f9605e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:03.918802 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81486a-2948-4b68-99cf-0b15d3f9605e None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:03.919320 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81486a-2948-4b68-99cf-0b15d3f9605e None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:03.919843 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81486a-2948-4b68-99cf-0b15d3f9605e None admin] PATH_INFO: `/v3/services` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:03.921400 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81486a-2948-4b68-99cf-0b15d3f9605e None admin] RBAC: Adding query filter params (type=network) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:03.921932 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81486a-2948-4b68-99cf-0b15d3f9605e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:03.923610 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81486a-2948-4b68-99cf-0b15d3f9605e None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:03.931939 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 401/827] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:16:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:03.932662 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 427/828] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Jul 7 15:16:03 2022] GET /identity/v3/services?type=network => generated 350 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:03.939610 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81486a-2948-4b68-99cf-0b15d3f9605e None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:03.942568 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b5618a-9578-42ed-af13-2c65f3d4db3b None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:04.076572 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:04.086578 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81486a-2948-4b68-99cf-0b15d3f9605e None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:04.115910 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:04.123196 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2996bd1c-009f-47fe-a163-50bcc31bd45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:04.123912 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996bd1c-009f-47fe-a163-50bcc31bd45a None admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:04.124199 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996bd1c-009f-47fe-a163-50bcc31bd45a None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:04.124481 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996bd1c-009f-47fe-a163-50bcc31bd45a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:04.125239 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2996bd1c-009f-47fe-a163-50bcc31bd45a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:04.126606 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2996bd1c-009f-47fe-a163-50bcc31bd45a None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:04.153931 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b5618a-9578-42ed-af13-2c65f3d4db3b None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:04.190119 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c664b354-7ce0-4168-a2ff-833e6535d721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:04.191035 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664b354-7ce0-4168-a2ff-833e6535d721 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:04.191332 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664b354-7ce0-4168-a2ff-833e6535d721 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:04.191635 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664b354-7ce0-4168-a2ff-833e6535d721 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:04.198752 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c664b354-7ce0-4168-a2ff-833e6535d721 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:04.200241 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c664b354-7ce0-4168-a2ff-833e6535d721 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:04.223102 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 428/829] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Jul 7 15:16:03 2022] POST /identity/v3/endpoints => generated 327 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:16:04.229802 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-c664b354-7ce0-4168-a2ff-833e6535d721 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:16:04.229802 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:04.229802 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:04.229802 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:04.229802 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:04.229802 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:04.229802 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:04.229802 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:04.229802 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:04.229802 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:04.229802 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:04.229802 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:04.229802 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 15:16:04.229802 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 15:16:04.229802 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 15:16:04.230748 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 15:16:04.230748 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:04.230748 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:04.230748 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:04.230748 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:04.230748 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:04.230748 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:04.230748 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:04.230748 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:04.230748 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:04.230748 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:04.230748 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:04.230748 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:04.230748 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:04.230748 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:04.230748 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:04.230748 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:04.231986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 15:16:04.231986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 15:16:04.231986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 15:16:04.231986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 15:16:04.231986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 15:16:04.231986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 15:16:04.231986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:16:04.231986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:04.231986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 402/830] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:16:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:04.237883 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2996bd1c-009f-47fe-a163-50bcc31bd45a None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:04.386128 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2996bd1c-009f-47fe-a163-50bcc31bd45a None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:04.437205 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a7a49-690d-4f71-ad98-aba63603c476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:04.437893 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a7a49-690d-4f71-ad98-aba63603c476 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:04.438178 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a7a49-690d-4f71-ad98-aba63603c476 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:04.438452 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a7a49-690d-4f71-ad98-aba63603c476 None admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:04.439326 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579a7a49-690d-4f71-ad98-aba63603c476 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:04.439665 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579a7a49-690d-4f71-ad98-aba63603c476 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:04.441198 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579a7a49-690d-4f71-ad98-aba63603c476 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:04.455749 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 429/831] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Jul 7 15:16:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:04.461833 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c664b354-7ce0-4168-a2ff-833e6535d721 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:04.573562 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c664b354-7ce0-4168-a2ff-833e6535d721 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:04.609047 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc753120-f1a8-45e5-9057-685729f1969a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:04.609710 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc753120-f1a8-45e5-9057-685729f1969a None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:04.610077 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc753120-f1a8-45e5-9057-685729f1969a None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:04.610329 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc753120-f1a8-45e5-9057-685729f1969a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:04.611177 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc753120-f1a8-45e5-9057-685729f1969a None admin] RBAC: Adding query filter params (role.id=0e7793fbf3064037984da651879e3a94, scope.project.id=c5260c7fd3614e9e93b167c86a94260e, user.id=de1632555cb345f2b9213da1925c02f4) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:04.611489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc753120-f1a8-45e5-9057-685729f1969a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:04.612944 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc753120-f1a8-45e5-9057-685729f1969a None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:04.615908 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:04.619631 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 403/832] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:16:04 2022] GET /identity/v3/role_assignments?user.id=de1632555cb345f2b9213da1925c02f4&scope.project.id=c5260c7fd3614e9e93b167c86a94260e&role.id=0e7793fbf3064037984da651879e3a94 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:04.664718 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f6c85-91ab-4428-b92a-33c537091143 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:04.665108 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f6c85-91ab-4428-b92a-33c537091143 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:04.665468 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f6c85-91ab-4428-b92a-33c537091143 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:04.667143 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 430/833] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:16:04.672873 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389c390-0ba4-4d0e-966b-537cb7ef5b9b None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:04.673240 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389c390-0ba4-4d0e-966b-537cb7ef5b9b None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:04.673593 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389c390-0ba4-4d0e-966b-537cb7ef5b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:04.692192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-2389c390-0ba4-4d0e-966b-537cb7ef5b9b None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:04.693540 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-2389c390-0ba4-4d0e-966b-537cb7ef5b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:04.771108 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 404/834] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:04 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:16:04.795465 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8f6c85-91ab-4428-b92a-33c537091143 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:04.798579 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:04.907009 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8f6c85-91ab-4428-b92a-33c537091143 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:04.944286 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb46c28-ab05-4506-935b-3ee12c09d665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:04.945105 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb46c28-ab05-4506-935b-3ee12c09d665 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:04.945464 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb46c28-ab05-4506-935b-3ee12c09d665 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:04.945809 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb46c28-ab05-4506-935b-3ee12c09d665 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:04.957882 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb46c28-ab05-4506-935b-3ee12c09d665 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:16:04.966034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-9bb46c28-ab05-4506-935b-3ee12c09d665 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:16:04.966034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:04.966034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:04.966034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:04.966034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:04.966034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:04.966034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:04.966034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:04.966034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:04.966034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:04.966034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:04.966034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:04.966034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:16:04.966034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:16:04.966034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:16:04.966034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:16:04.967243 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:16:04.967243 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:16:04.967243 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:16:04.967243 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:16:04.967243 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:04.967243 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:04.967243 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:04.967243 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:04.967243 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:04.967243 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:04.967243 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:04.967243 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:04.967243 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:04.967243 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:04.967243 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:04.967243 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:04.968384 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:04.968384 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:04.968384 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:04.968384 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:04.968384 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:16:04.968384 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:16:04.968384 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:16:04.968384 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:16:04.968384 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:16:04.968384 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:16:04.968384 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:16:04.968384 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:04.969264 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 431/835] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Jul 7 15:16:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:04.974832 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2389c390-0ba4-4d0e-966b-537cb7ef5b9b None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:05.078436 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2389c390-0ba4-4d0e-966b-537cb7ef5b9b None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:05.108416 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:05.113349 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd7e06-3d12-428f-a384-be795654225c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:05.114071 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd7e06-3d12-428f-a384-be795654225c None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:05.114358 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd7e06-3d12-428f-a384-be795654225c None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:05.114687 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd7e06-3d12-428f-a384-be795654225c None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:05.115444 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fd7e06-3d12-428f-a384-be795654225c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:05.115781 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fd7e06-3d12-428f-a384-be795654225c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:05.117146 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fd7e06-3d12-428f-a384-be795654225c None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:05.126271 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 405/836] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:16:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:05.131866 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb46c28-ab05-4506-935b-3ee12c09d665 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:05.199824 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:05.242853 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb46c28-ab05-4506-935b-3ee12c09d665 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:05.279521 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24506e70-59f4-4b1c-bcb2-898fa6654f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:05.280356 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24506e70-59f4-4b1c-bcb2-898fa6654f2b None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:05.280695 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24506e70-59f4-4b1c-bcb2-898fa6654f2b None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:05.280981 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24506e70-59f4-4b1c-bcb2-898fa6654f2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:05.296340 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24506e70-59f4-4b1c-bcb2-898fa6654f2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:05.297793 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24506e70-59f4-4b1c-bcb2-898fa6654f2b None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:05.315495 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-24506e70-59f4-4b1c-bcb2-898fa6654f2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:05.315495 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:05.315495 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:05.315495 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:05.315495 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:05.315495 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:05.315495 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:05.315495 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:05.315495 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:05.315495 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:05.315495 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:05.315495 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:05.315495 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:16:05.315495 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:16:05.315495 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:16:05.316440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:16:05.316440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:05.316440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:05.316440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:05.316440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:05.316440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:05.316440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:05.316440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:05.316440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:05.316440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:05.316440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:05.316440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:05.316440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:05.316440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:05.316440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:05.316440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:05.316440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:05.317760 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:16:05.317760 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:16:05.317760 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:05.317760 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:05.317760 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 432/837] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:16:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:05.322552 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd7e06-3d12-428f-a384-be795654225c None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:05.431532 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd7e06-3d12-428f-a384-be795654225c None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:05.466779 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee979ba4-94be-4cf3-8e6c-0fcd70cb36bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:05.467387 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee979ba4-94be-4cf3-8e6c-0fcd70cb36bd None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:05.467698 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee979ba4-94be-4cf3-8e6c-0fcd70cb36bd None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:05.467975 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee979ba4-94be-4cf3-8e6c-0fcd70cb36bd None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:05.468696 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee979ba4-94be-4cf3-8e6c-0fcd70cb36bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:05.469000 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee979ba4-94be-4cf3-8e6c-0fcd70cb36bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:05.470356 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee979ba4-94be-4cf3-8e6c-0fcd70cb36bd None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:05.484150 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 406/838] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:16:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:05.489444 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24506e70-59f4-4b1c-bcb2-898fa6654f2b None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:05.499781 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a2f74-e944-4a52-b25b-e8eef909b7db None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:05.500071 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a2f74-e944-4a52-b25b-e8eef909b7db None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:05.500354 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a2f74-e944-4a52-b25b-e8eef909b7db None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:05.505459 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 407/839] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:16:05.507527 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b748ce-c72b-4378-a5e2-9e105f975f06 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:05.507862 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b748ce-c72b-4378-a5e2-9e105f975f06 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:05.508141 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b748ce-c72b-4378-a5e2-9e105f975f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:05.529882 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-c9b748ce-c72b-4378-a5e2-9e105f975f06 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:05.531943 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-c9b748ce-c72b-4378-a5e2-9e105f975f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:05.600041 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:05.611757 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 408/840] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:05 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:16:05.631966 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b748ce-c72b-4378-a5e2-9e105f975f06 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:05.747888 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b748ce-c72b-4378-a5e2-9e105f975f06 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:05.755554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24506e70-59f4-4b1c-bcb2-898fa6654f2b None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:05.775187 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:05.784303 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfac3e7-8eba-47b7-a41d-f8602659d0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:05.785188 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfac3e7-8eba-47b7-a41d-f8602659d0bc None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:05.785523 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfac3e7-8eba-47b7-a41d-f8602659d0bc None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:05.790728 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfac3e7-8eba-47b7-a41d-f8602659d0bc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:05.797695 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfac3e7-8eba-47b7-a41d-f8602659d0bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:16:05.800763 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a047a5-6fd7-4815-9175-7d84c11a6663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:05.801551 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a047a5-6fd7-4815-9175-7d84c11a6663 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:05.801831 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a047a5-6fd7-4815-9175-7d84c11a6663 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:05.802104 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a047a5-6fd7-4815-9175-7d84c11a6663 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:05.806075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-4dfac3e7-8eba-47b7-a41d-f8602659d0bc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:16:05.806075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:05.806075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:05.806075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:05.806075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:05.806075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:05.806075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:05.806075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:05.806075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:05.806075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:05.806075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:05.806075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:05.806075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:16:05.806075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:16:05.806075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:16:05.806075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:16:05.807095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:16:05.807095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:16:05.807095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:16:05.807095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:16:05.807095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:05.807095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:05.807095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:05.807095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:05.807095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:05.807095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:05.807095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:05.807095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:05.807095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:05.807095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:05.807095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:05.807095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:05.808047 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:05.808047 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:05.808047 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:05.808047 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:05.808047 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:16:05.808047 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:16:05.808047 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:16:05.808047 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:16:05.808047 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:16:05.808047 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:16:05.808047 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:16:05.808047 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:05.811598 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a047a5-6fd7-4815-9175-7d84c11a6663 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:05.811598 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a047a5-6fd7-4815-9175-7d84c11a6663 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:05.812857 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 409/841] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Jul 7 15:16:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:16:05.814687 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfac3e7-8eba-47b7-a41d-f8602659d0bc None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:05.827654 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-47a047a5-6fd7-4815-9175-7d84c11a6663 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 07 15:16:05.827654 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:05.827654 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:05.827654 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:05.827654 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:05.827654 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:05.827654 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:05.827654 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:05.827654 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:05.827654 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:05.827654 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:05.827654 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:05.827654 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 15:16:05.827654 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 15:16:05.827654 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 15:16:05.827654 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 15:16:05.829513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:05.829513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:05.829513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 15:16:05.829513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:16:05.829513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 15:16:05.829513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:16:05.829513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:05.829513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:05.829513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:05.829513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:05.829513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:05.829513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:05.829513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:05.829513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:05.829513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:05.829513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:05.829513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:05.829513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:05.830660 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:05.830660 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:05.830660 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 15:16:05.830660 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 15:16:05.830660 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 15:16:05.830660 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 15:16:05.830660 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 15:16:05.830660 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 15:16:05.830660 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 07 15:16:05.830660 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:05.830660 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 433/842] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Jul 7 15:16:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:05.836185 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a047a5-6fd7-4815-9175-7d84c11a6663 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:05.873120 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:05.949409 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfac3e7-8eba-47b7-a41d-f8602659d0bc None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:05.953900 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a047a5-6fd7-4815-9175-7d84c11a6663 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:05.995014 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeefae7-c6cb-467d-a791-986c9611cd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:05.995507 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeefae7-c6cb-467d-a791-986c9611cd46 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:05.995832 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeefae7-c6cb-467d-a791-986c9611cd46 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:05.996133 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeefae7-c6cb-467d-a791-986c9611cd46 None admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:05.996974 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeefae7-c6cb-467d-a791-986c9611cd46 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:05.997282 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeefae7-c6cb-467d-a791-986c9611cd46 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:05.998798 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeefae7-c6cb-467d-a791-986c9611cd46 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:06.012050 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95689a20-b536-469f-8365-3ac775e0901d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:06.012961 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95689a20-b536-469f-8365-3ac775e0901d None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:06.013642 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95689a20-b536-469f-8365-3ac775e0901d None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:06.015521 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95689a20-b536-469f-8365-3ac775e0901d None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:06.016515 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95689a20-b536-469f-8365-3ac775e0901d None admin] RBAC: Adding query filter params (name=service) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:06.016836 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95689a20-b536-469f-8365-3ac775e0901d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:06.018197 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95689a20-b536-469f-8365-3ac775e0901d None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:06.027213 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 434/843] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Jul 7 15:16:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:06.027213 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92674d-422b-449f-9e32-88c7728d6563 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:06.027213 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92674d-422b-449f-9e32-88c7728d6563 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:06.027213 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92674d-422b-449f-9e32-88c7728d6563 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:06.027213 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 435/844] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:16:06.028349 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e92674d-422b-449f-9e32-88c7728d6563 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:06.034623 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 410/845] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Jul 7 15:16:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:06.034623 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151e995-3690-4ee9-bbca-fe3256cc0668 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:06.034623 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151e995-3690-4ee9-bbca-fe3256cc0668 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:06.034623 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151e995-3690-4ee9-bbca-fe3256cc0668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:06.050196 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-7151e995-3690-4ee9-bbca-fe3256cc0668 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:06.051364 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-7151e995-3690-4ee9-bbca-fe3256cc0668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:06.065366 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:06.120308 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 411/846] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:06 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:16:06.121031 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7151e995-3690-4ee9-bbca-fe3256cc0668 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:06.150608 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e92674d-422b-449f-9e32-88c7728d6563 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:06.158897 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:06.187166 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c6813-4758-4bf0-a95b-8dda5d90c4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:06.187873 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c6813-4758-4bf0-a95b-8dda5d90c4c1 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:06.188192 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c6813-4758-4bf0-a95b-8dda5d90c4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:06.189148 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c6813-4758-4bf0-a95b-8dda5d90c4c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:06.228761 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7151e995-3690-4ee9-bbca-fe3256cc0668 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:06.261786 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116c6813-4758-4bf0-a95b-8dda5d90c4c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:06.264179 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116c6813-4758-4bf0-a95b-8dda5d90c4c1 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:06.265068 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a222a-f929-4a31-aa69-5e609495524e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:06.265862 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a222a-f929-4a31-aa69-5e609495524e None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:06.266147 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a222a-f929-4a31-aa69-5e609495524e None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:06.266421 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a222a-f929-4a31-aa69-5e609495524e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:06.276775 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a222a-f929-4a31-aa69-5e609495524e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:06.278197 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a222a-f929-4a31-aa69-5e609495524e None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:06.296667 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-174a222a-f929-4a31-aa69-5e609495524e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:06.296667 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:06.296667 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:06.296667 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:06.296667 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:06.296667 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:06.296667 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:06.296667 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:06.296667 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:06.296667 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:06.296667 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:06.296667 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:06.296667 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:16:06.296667 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:16:06.296667 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:16:06.297600 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:16:06.297600 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:06.297600 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:06.297600 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:06.297600 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:06.297600 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:06.297600 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:06.297600 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:06.297600 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:06.297600 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:06.297600 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:06.297600 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:06.297600 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:06.297600 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:06.297600 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:06.297600 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:06.297600 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:06.298506 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:16:06.298506 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:16:06.298506 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:06.298506 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:06.298506 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 412/847] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:16:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:06.299148 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a222a-f929-4a31-aa69-5e609495524e None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:06.304294 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:06.380949 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 436/848] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:16:06 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 07 15:16:06.381786 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c6813-4758-4bf0-a95b-8dda5d90c4c1 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:06.395455 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:06.413882 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a222a-f929-4a31-aa69-5e609495524e None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:06.450875 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f46c0-ee12-4459-9d47-600ec4fbf127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:06.452259 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f46c0-ee12-4459-9d47-600ec4fbf127 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:06.452544 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f46c0-ee12-4459-9d47-600ec4fbf127 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:06.452818 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f46c0-ee12-4459-9d47-600ec4fbf127 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:06.453463 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669f46c0-ee12-4459-9d47-600ec4fbf127 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:06.454814 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669f46c0-ee12-4459-9d47-600ec4fbf127 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:06.467246 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-669f46c0-ee12-4459-9d47-600ec4fbf127 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 07 15:16:06.467246 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:06.467246 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 15:16:06.467246 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 15:16:06.467246 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 15:16:06.467246 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 15:16:06.467246 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 15:16:06.467246 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:16:06.467246 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 07 15:16:06.467246 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:06.467246 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 15:16:06.467246 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:06.467246 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:06.467246 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:06.467246 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:06.467246 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:06.467246 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:06.468317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:06.468317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:06.468317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:06.468317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:06.468317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:06.468317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:06.468317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 15:16:06.468317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 15:16:06.468317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 15:16:06.468317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 15:16:06.468317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:06.468317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:06.468317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:06.468317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:06.468317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:06.468317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:06.469190 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:06.469190 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:06.469190 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:06.469190 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:06.469190 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:06.469190 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:06.469190 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:06.469190 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:06.469190 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:06.469190 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:06.469190 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 15:16:06.469190 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:16:06.469190 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 07 15:16:06.469190 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:06.469190 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 413/849] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Jul 7 15:16:06 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:06.470025 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f46c0-ee12-4459-9d47-600ec4fbf127 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:06.492706 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c6813-4758-4bf0-a95b-8dda5d90c4c1 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:06.522924 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:06.527714 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b38d7ff-a82a-4011-8790-3393315a922e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:06.528372 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38d7ff-a82a-4011-8790-3393315a922e None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:06.528649 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38d7ff-a82a-4011-8790-3393315a922e None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:06.528918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38d7ff-a82a-4011-8790-3393315a922e None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:06.529664 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b38d7ff-a82a-4011-8790-3393315a922e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:06.536451 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b38d7ff-a82a-4011-8790-3393315a922e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:06.536451 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b38d7ff-a82a-4011-8790-3393315a922e None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:06.543144 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 437/850] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:16:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:06.544200 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b38d7ff-a82a-4011-8790-3393315a922e None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:06.583205 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f46c0-ee12-4459-9d47-600ec4fbf127 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:06.618723 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ce76d-4d74-4fed-a6ae-30077632a4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:06.619556 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ce76d-4d74-4fed-a6ae-30077632a4dd None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:06.619872 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ce76d-4d74-4fed-a6ae-30077632a4dd None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:06.620192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ce76d-4d74-4fed-a6ae-30077632a4dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:06.633004 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15ce76d-4d74-4fed-a6ae-30077632a4dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:06.634444 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15ce76d-4d74-4fed-a6ae-30077632a4dd None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:06.648051 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b38d7ff-a82a-4011-8790-3393315a922e None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:06.652613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-b15ce76d-4d74-4fed-a6ae-30077632a4dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:06.652613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:06.652613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:06.652613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:06.652613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:06.652613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:06.652613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:06.652613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:06.652613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:06.652613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:06.652613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:06.652613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:06.652613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:16:06.652613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:16:06.652613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:16:06.653579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:16:06.653579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:06.653579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:06.653579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:06.653579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:06.653579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:06.653579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:06.653579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:06.653579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:06.653579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:06.653579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:06.653579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:06.653579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:06.653579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:06.653579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:06.653579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:06.653579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:06.655095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:16:06.655095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:16:06.655095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:06.655095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:06.655095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 414/851] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:16:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:06.655095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ce76d-4d74-4fed-a6ae-30077632a4dd None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:06.657939 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:06.689127 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec70ef-6a56-4f1e-a9b8-ec40de5bbf3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:06.689790 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec70ef-6a56-4f1e-a9b8-ec40de5bbf3f None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:06.690068 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec70ef-6a56-4f1e-a9b8-ec40de5bbf3f None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:06.690337 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec70ef-6a56-4f1e-a9b8-ec40de5bbf3f None admin] PATH_INFO: `/v3/services` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:06.691042 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ec70ef-6a56-4f1e-a9b8-ec40de5bbf3f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:06.691339 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ec70ef-6a56-4f1e-a9b8-ec40de5bbf3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:06.692739 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ec70ef-6a56-4f1e-a9b8-ec40de5bbf3f None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:06.702284 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 438/852] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Jul 7 15:16:06 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:06.702284 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec70ef-6a56-4f1e-a9b8-ec40de5bbf3f None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:06.761080 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:06.768513 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ce76d-4d74-4fed-a6ae-30077632a4dd None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:06.805636 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b4b03-c439-4ebb-9ddb-cf4cb703a2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:06.806327 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b4b03-c439-4ebb-9ddb-cf4cb703a2e4 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:06.806643 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b4b03-c439-4ebb-9ddb-cf4cb703a2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:06.806915 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b4b03-c439-4ebb-9ddb-cf4cb703a2e4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:06.808757 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec70ef-6a56-4f1e-a9b8-ec40de5bbf3f None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:06.815495 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59b4b03-c439-4ebb-9ddb-cf4cb703a2e4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:06.816973 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59b4b03-c439-4ebb-9ddb-cf4cb703a2e4 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:06.831670 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-a59b4b03-c439-4ebb-9ddb-cf4cb703a2e4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 07 15:16:06.831670 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:06.831670 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:06.831670 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:06.831670 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:06.831670 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:06.831670 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:06.831670 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:06.831670 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:06.831670 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:06.831670 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:06.831670 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:06.831670 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 15:16:06.831670 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 15:16:06.831670 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 15:16:06.831670 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 15:16:06.832653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:06.832653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:06.832653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 15:16:06.832653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:16:06.832653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 15:16:06.832653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:16:06.832653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:06.832653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:06.832653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:06.832653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:06.832653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:06.832653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:06.832653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:06.832653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:06.832653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:06.832653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:06.832653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:06.832653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:06.833631 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:06.833631 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:06.833631 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 15:16:06.833631 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 15:16:06.833631 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 15:16:06.833631 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 15:16:06.833631 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 15:16:06.833631 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 15:16:06.833631 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 07 15:16:06.833631 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:06.833631 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 415/853] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Jul 7 15:16:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:06.833631 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b4b03-c439-4ebb-9ddb-cf4cb703a2e4 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:06.848843 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c2ab5-be86-4d67-b1b9-56325ca873d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:06.849504 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c2ab5-be86-4d67-b1b9-56325ca873d1 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:06.849784 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c2ab5-be86-4d67-b1b9-56325ca873d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:06.850058 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c2ab5-be86-4d67-b1b9-56325ca873d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:06.850780 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c2ab5-be86-4d67-b1b9-56325ca873d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:06.851086 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c2ab5-be86-4d67-b1b9-56325ca873d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:06.852488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c2ab5-be86-4d67-b1b9-56325ca873d1 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:06.863941 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 439/854] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:16:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:06.865145 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44620d3-d2c6-42b4-aa2a-0adb9741d72a None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:06.865437 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44620d3-d2c6-42b4-aa2a-0adb9741d72a None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:06.865710 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44620d3-d2c6-42b4-aa2a-0adb9741d72a None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:06.866910 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 440/855] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:16:06.867610 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44620d3-d2c6-42b4-aa2a-0adb9741d72a None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:06.898623 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:06.948439 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b4b03-c439-4ebb-9ddb-cf4cb703a2e4 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:06.972427 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44620d3-d2c6-42b4-aa2a-0adb9741d72a None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:06.986232 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71021f32-a8e9-40f2-9990-f668745b93e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:06.986891 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71021f32-a8e9-40f2-9990-f668745b93e5 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:06.987174 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71021f32-a8e9-40f2-9990-f668745b93e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:06.987445 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71021f32-a8e9-40f2-9990-f668745b93e5 None admin] PATH_INFO: `/v3/services` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:06.988183 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71021f32-a8e9-40f2-9990-f668745b93e5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:06.988490 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71021f32-a8e9-40f2-9990-f668745b93e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:06.989283 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:06.989839 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71021f32-a8e9-40f2-9990-f668745b93e5 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:06.994343 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 416/856] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Jul 7 15:16:06 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:06.995110 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71021f32-a8e9-40f2-9990-f668745b93e5 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:07.008819 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d77fe-a73b-412d-b585-6793d7a25049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:07.010494 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d77fe-a73b-412d-b585-6793d7a25049 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:07.010787 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d77fe-a73b-412d-b585-6793d7a25049 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:07.011107 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d77fe-a73b-412d-b585-6793d7a25049 None admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:07.011998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9d77fe-a73b-412d-b585-6793d7a25049 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:07.012314 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9d77fe-a73b-412d-b585-6793d7a25049 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:07.013795 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9d77fe-a73b-412d-b585-6793d7a25049 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:07.034567 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 441/857] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Jul 7 15:16:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:07.036241 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0403146-afa6-4099-b0ed-941a5e165882 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:07.036733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0403146-afa6-4099-b0ed-941a5e165882 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:07.037235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0403146-afa6-4099-b0ed-941a5e165882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:07.056914 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-e0403146-afa6-4099-b0ed-941a5e165882 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:07.058060 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-e0403146-afa6-4099-b0ed-941a5e165882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:07.129720 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 442/858] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:07 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:16:07.130678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0403146-afa6-4099-b0ed-941a5e165882 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:07.131802 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71021f32-a8e9-40f2-9990-f668745b93e5 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:07.167508 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:07.179506 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2742af-ee88-44ee-8444-dd64f0962597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:07.180341 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2742af-ee88-44ee-8444-dd64f0962597 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:07.180766 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2742af-ee88-44ee-8444-dd64f0962597 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:07.184013 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2742af-ee88-44ee-8444-dd64f0962597 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:07.189254 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2742af-ee88-44ee-8444-dd64f0962597 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:07.191279 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2742af-ee88-44ee-8444-dd64f0962597 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:07.209878 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-ec2742af-ee88-44ee-8444-dd64f0962597 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:16:07.209878 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:07.209878 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:07.209878 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:07.209878 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:07.209878 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:07.209878 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:07.209878 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:07.209878 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:07.209878 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:07.209878 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:07.209878 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:07.209878 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 15:16:07.209878 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 15:16:07.209878 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 15:16:07.211002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 15:16:07.211002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:07.211002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:07.211002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:07.211002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:07.211002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:07.211002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:07.211002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:07.211002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:07.211002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:07.211002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:07.211002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:07.211002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:07.211002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:07.211002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:07.211002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:07.211002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:07.211970 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 15:16:07.211970 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 15:16:07.211970 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 15:16:07.211970 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 15:16:07.211970 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 15:16:07.211970 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 15:16:07.211970 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:16:07.211970 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:07.211970 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 417/859] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:16:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:07.211970 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2742af-ee88-44ee-8444-dd64f0962597 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:07.253884 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0403146-afa6-4099-b0ed-941a5e165882 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:07.257527 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:07.292908 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b153d84c-5ef9-4c5e-9df1-978532f2933c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:07.293659 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153d84c-5ef9-4c5e-9df1-978532f2933c None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:07.293934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153d84c-5ef9-4c5e-9df1-978532f2933c None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:07.294203 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153d84c-5ef9-4c5e-9df1-978532f2933c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:07.329345 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2742af-ee88-44ee-8444-dd64f0962597 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:07.366537 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b153d84c-5ef9-4c5e-9df1-978532f2933c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:07.367144 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c89aca-3533-4ec0-9c63-1e4da5eaf7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:07.367991 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c89aca-3533-4ec0-9c63-1e4da5eaf7d5 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:07.368118 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b153d84c-5ef9-4c5e-9df1-978532f2933c None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:07.368279 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c89aca-3533-4ec0-9c63-1e4da5eaf7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:07.368570 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c89aca-3533-4ec0-9c63-1e4da5eaf7d5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:07.369220 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c89aca-3533-4ec0-9c63-1e4da5eaf7d5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:07.371233 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c89aca-3533-4ec0-9c63-1e4da5eaf7d5 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:07.382995 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-36c89aca-3533-4ec0-9c63-1e4da5eaf7d5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 07 15:16:07.382995 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:07.382995 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 15:16:07.382995 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 15:16:07.382995 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 15:16:07.382995 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 15:16:07.382995 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 15:16:07.382995 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:16:07.382995 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 07 15:16:07.382995 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:07.382995 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 15:16:07.382995 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:07.382995 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:07.382995 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:07.382995 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:07.382995 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:07.383983 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:07.383983 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:07.383983 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:07.383983 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:07.383983 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:07.383983 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:07.383983 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:07.383983 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 15:16:07.383983 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 15:16:07.383983 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 15:16:07.383983 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 15:16:07.383983 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:07.383983 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:07.383983 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:07.383983 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:07.383983 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:07.384913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:07.384913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:07.384913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:07.384913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:07.384913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:07.384913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:07.384913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:07.384913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:07.384913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:07.384913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:07.384913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:07.384913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 15:16:07.384913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:16:07.384913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 07 15:16:07.384913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:07.384913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 418/860] 10.222.0.25 () {56 vars in 1132 bytes} [Thu Jul 7 15:16:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:07.385754 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c89aca-3533-4ec0-9c63-1e4da5eaf7d5 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:07.439319 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:07.481388 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 443/861] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:16:07 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 07 15:16:07.482235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b153d84c-5ef9-4c5e-9df1-978532f2933c None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:07.487851 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:07.498812 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c89aca-3533-4ec0-9c63-1e4da5eaf7d5 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:07.533965 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c44bb-369f-43d0-b5e9-7b6d4677d726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:07.534620 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c44bb-369f-43d0-b5e9-7b6d4677d726 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:07.534899 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c44bb-369f-43d0-b5e9-7b6d4677d726 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:07.535168 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c44bb-369f-43d0-b5e9-7b6d4677d726 None admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:07.535986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c44bb-369f-43d0-b5e9-7b6d4677d726 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:07.536289 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c44bb-369f-43d0-b5e9-7b6d4677d726 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:07.537690 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c44bb-369f-43d0-b5e9-7b6d4677d726 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:07.554292 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 419/862] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Jul 7 15:16:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:07.554292 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c44bb-369f-43d0-b5e9-7b6d4677d726 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:07.602642 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b153d84c-5ef9-4c5e-9df1-978532f2933c None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:07.643139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea3ea38-d51b-4cd9-98d7-536e1f934b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:07.643830 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea3ea38-d51b-4cd9-98d7-536e1f934b4d None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:07.644219 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea3ea38-d51b-4cd9-98d7-536e1f934b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:07.644416 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea3ea38-d51b-4cd9-98d7-536e1f934b4d None admin] PATH_INFO: `/v3/services` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:07.645077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea3ea38-d51b-4cd9-98d7-536e1f934b4d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:07.645574 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea3ea38-d51b-4cd9-98d7-536e1f934b4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:07.646733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea3ea38-d51b-4cd9-98d7-536e1f934b4d None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:07.654136 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c44bb-369f-43d0-b5e9-7b6d4677d726 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:07.657682 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 444/863] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Jul 7 15:16:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:07.657805 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea3ea38-d51b-4cd9-98d7-536e1f934b4d None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:07.666872 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:07.691159 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe7fe83-90e6-4e74-a562-4df37a9366fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:07.691979 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7fe83-90e6-4e74-a562-4df37a9366fa None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:07.692261 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7fe83-90e6-4e74-a562-4df37a9366fa None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:07.692534 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7fe83-90e6-4e74-a562-4df37a9366fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:07.704130 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe7fe83-90e6-4e74-a562-4df37a9366fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:07.705882 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe7fe83-90e6-4e74-a562-4df37a9366fa None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:07.724112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-dfe7fe83-90e6-4e74-a562-4df37a9366fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:07.724112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:07.724112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:07.724112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:07.724112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:07.724112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:07.724112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:07.724112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:07.724112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:07.724112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:07.724112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:07.724112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:07.724112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:16:07.724112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:16:07.724112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:16:07.725027 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:16:07.725027 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:07.725027 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:07.725027 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:07.725027 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:07.725027 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:07.725027 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:07.725027 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:07.725027 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:07.725027 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:07.725027 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:07.725027 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:07.725027 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:07.725027 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:07.725027 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:07.725027 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:07.725027 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:07.725952 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:16:07.725952 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:16:07.725952 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:07.725952 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:07.725952 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 420/864] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:16:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:07.726240 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe7fe83-90e6-4e74-a562-4df37a9366fa None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:07.736367 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:07.774340 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea3ea38-d51b-4cd9-98d7-536e1f934b4d None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:07.812169 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65319da8-8fe4-4fa2-8617-02138de96d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:07.812939 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65319da8-8fe4-4fa2-8617-02138de96d86 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:07.813214 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65319da8-8fe4-4fa2-8617-02138de96d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:07.813494 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65319da8-8fe4-4fa2-8617-02138de96d86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:07.814319 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65319da8-8fe4-4fa2-8617-02138de96d86 None admin] RBAC: Adding query filter params (role.id=0e7793fbf3064037984da651879e3a94, scope.project.id=c5260c7fd3614e9e93b167c86a94260e, user.id=b18033f849b240999d12c1885d2f7b91) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:07.814629 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65319da8-8fe4-4fa2-8617-02138de96d86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:07.816076 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65319da8-8fe4-4fa2-8617-02138de96d86 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:07.821833 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 445/865] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:16:07 2022] GET /identity/v3/role_assignments?user.id=b18033f849b240999d12c1885d2f7b91&scope.project.id=c5260c7fd3614e9e93b167c86a94260e&role.id=0e7793fbf3064037984da651879e3a94 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:07.822589 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65319da8-8fe4-4fa2-8617-02138de96d86 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:07.845102 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe7fe83-90e6-4e74-a562-4df37a9366fa None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:07.886108 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8801cd6f-7365-47b8-863b-6e516c79174f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:07.887242 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801cd6f-7365-47b8-863b-6e516c79174f None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:07.887784 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801cd6f-7365-47b8-863b-6e516c79174f None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:07.888254 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801cd6f-7365-47b8-863b-6e516c79174f None admin] PATH_INFO: `/v3/services` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:07.889753 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8801cd6f-7365-47b8-863b-6e516c79174f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:07.890392 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8801cd6f-7365-47b8-863b-6e516c79174f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:07.892134 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8801cd6f-7365-47b8-863b-6e516c79174f None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:07.898370 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 421/866] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Jul 7 15:16:07 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:07.969551 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65319da8-8fe4-4fa2-8617-02138de96d86 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:08.008330 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:08.012312 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afdb81f-0557-4a33-8660-8f8e728f1a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:08.013400 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afdb81f-0557-4a33-8660-8f8e728f1a4d None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:08.014330 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afdb81f-0557-4a33-8660-8f8e728f1a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:08.014638 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afdb81f-0557-4a33-8660-8f8e728f1a4d None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:08.015390 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afdb81f-0557-4a33-8660-8f8e728f1a4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:08.015731 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afdb81f-0557-4a33-8660-8f8e728f1a4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:08.017162 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afdb81f-0557-4a33-8660-8f8e728f1a4d None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:08.030853 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 446/867] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:16:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:08.036380 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8801cd6f-7365-47b8-863b-6e516c79174f None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:08.070642 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:08.149640 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8801cd6f-7365-47b8-863b-6e516c79174f None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:08.184496 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc405570-e088-4285-bac9-0088fadcd9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:08.185282 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc405570-e088-4285-bac9-0088fadcd9b6 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:08.186092 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc405570-e088-4285-bac9-0088fadcd9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:08.186365 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc405570-e088-4285-bac9-0088fadcd9b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:08.193520 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc405570-e088-4285-bac9-0088fadcd9b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:08.194947 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc405570-e088-4285-bac9-0088fadcd9b6 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:08.209288 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-bc405570-e088-4285-bac9-0088fadcd9b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:16:08.209288 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:08.209288 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:08.209288 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:08.209288 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:08.209288 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:08.209288 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:08.209288 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:08.209288 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:08.209288 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:08.209288 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:08.209288 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:08.209288 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 15:16:08.209288 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 15:16:08.209288 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 15:16:08.210220 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 15:16:08.210220 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:08.210220 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:08.210220 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:08.210220 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:08.210220 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:08.210220 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:08.210220 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:08.210220 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:08.210220 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:08.210220 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:08.210220 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:08.210220 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:08.210220 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:08.210220 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:08.210220 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:08.210220 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:08.211557 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 15:16:08.211557 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 15:16:08.211557 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 15:16:08.211557 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 15:16:08.211557 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 15:16:08.211557 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 15:16:08.211557 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:16:08.211557 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:08.211557 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 422/868] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:16:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:08.216365 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afdb81f-0557-4a33-8660-8f8e728f1a4d None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:08.327888 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afdb81f-0557-4a33-8660-8f8e728f1a4d None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:08.363676 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff561db5-f7aa-4f14-a366-dcfe04ab8f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:08.364672 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff561db5-f7aa-4f14-a366-dcfe04ab8f61 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:08.365191 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff561db5-f7aa-4f14-a366-dcfe04ab8f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:08.365674 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff561db5-f7aa-4f14-a366-dcfe04ab8f61 None admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:08.366663 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff561db5-f7aa-4f14-a366-dcfe04ab8f61 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:08.367185 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff561db5-f7aa-4f14-a366-dcfe04ab8f61 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:08.368840 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff561db5-f7aa-4f14-a366-dcfe04ab8f61 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:08.381778 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 447/869] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Jul 7 15:16:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:08.391588 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc405570-e088-4285-bac9-0088fadcd9b6 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:08.480889 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:08.496719 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc405570-e088-4285-bac9-0088fadcd9b6 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:08.531837 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47342a03-f037-41d4-8a19-2cf798f2803e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:08.532493 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47342a03-f037-41d4-8a19-2cf798f2803e None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:08.532778 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47342a03-f037-41d4-8a19-2cf798f2803e None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:08.533055 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47342a03-f037-41d4-8a19-2cf798f2803e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:08.533967 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47342a03-f037-41d4-8a19-2cf798f2803e None admin] RBAC: Adding query filter params (role.id=3d4f4e0a472e4ff9ac68d90c8612e956, scope.project.id=c5260c7fd3614e9e93b167c86a94260e, user.id=cd3c04f39d774998b66e6ce920925691) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:08.534279 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47342a03-f037-41d4-8a19-2cf798f2803e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:08.535723 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47342a03-f037-41d4-8a19-2cf798f2803e None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:08.541999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 423/870] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:16:08 2022] GET /identity/v3/role_assignments?user.id=cd3c04f39d774998b66e6ce920925691&scope.project.id=c5260c7fd3614e9e93b167c86a94260e&role.id=3d4f4e0a472e4ff9ac68d90c8612e956 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:09.068015 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aef1e3-f14c-4b56-a0a3-756d5f0a06b2 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:09.068532 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aef1e3-f14c-4b56-a0a3-756d5f0a06b2 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:09.068610 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aef1e3-f14c-4b56-a0a3-756d5f0a06b2 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:09.070254 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 448/871] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:16:09.075982 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099dd46-8d7d-4416-b3d9-daebb65fc3be None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:09.077024 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099dd46-8d7d-4416-b3d9-daebb65fc3be None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:09.077103 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099dd46-8d7d-4416-b3d9-daebb65fc3be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:09.096041 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-5099dd46-8d7d-4416-b3d9-daebb65fc3be None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:09.097251 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-5099dd46-8d7d-4416-b3d9-daebb65fc3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:09.173671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 424/872] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:09 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:16:09.195358 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aef1e3-f14c-4b56-a0a3-756d5f0a06b2 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:09.265503 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:09.315394 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aef1e3-f14c-4b56-a0a3-756d5f0a06b2 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:09.351435 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b9047b-e161-4f04-ad16-277bf1a10552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:09.352599 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9047b-e161-4f04-ad16-277bf1a10552 None admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:09.353078 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9047b-e161-4f04-ad16-277bf1a10552 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:09.353563 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9047b-e161-4f04-ad16-277bf1a10552 None admin] PATH_INFO: `/v3/services` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:09.354480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b9047b-e161-4f04-ad16-277bf1a10552 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:09.356221 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b9047b-e161-4f04-ad16-277bf1a10552 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:09.419462 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 449/873] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Jul 7 15:16:09 2022] POST /identity/v3/services => generated 251 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:16:10.044404 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87ecdb-d17e-4bc9-bb34-c42f45788c0f None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:10.045251 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87ecdb-d17e-4bc9-bb34-c42f45788c0f None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:10.045508 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87ecdb-d17e-4bc9-bb34-c42f45788c0f None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:10.047189 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 425/874] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:16:10.052252 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97883cf-715b-4114-bc7d-2bfb1b8d5818 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:10.052371 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ddf8d-4ad3-46b1-ab16-571b72ee3b39 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:10.052558 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97883cf-715b-4114-bc7d-2bfb1b8d5818 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:10.052656 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ddf8d-4ad3-46b1-ab16-571b72ee3b39 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:10.052816 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97883cf-715b-4114-bc7d-2bfb1b8d5818 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:10.052907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ddf8d-4ad3-46b1-ab16-571b72ee3b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:10.054371 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 426/875] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:16:10.059293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50b793-78ad-497b-995d-2454d1cd6aca None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:10.063594 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50b793-78ad-497b-995d-2454d1cd6aca None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:10.063594 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50b793-78ad-497b-995d-2454d1cd6aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:10.071351 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-074ddf8d-4ad3-46b1-ab16-571b72ee3b39 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:10.072602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-074ddf8d-4ad3-46b1-ab16-571b72ee3b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:10.077295 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-3f50b793-78ad-497b-995d-2454d1cd6aca None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:10.078465 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-3f50b793-78ad-497b-995d-2454d1cd6aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:10.150993 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 450/876] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:10 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:16:10.152392 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 427/877] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:10 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 07 15:16:10.174869 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50b793-78ad-497b-995d-2454d1cd6aca None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:10.175037 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ddf8d-4ad3-46b1-ab16-571b72ee3b39 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:10.187137 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:10.229155 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:10.285009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50b793-78ad-497b-995d-2454d1cd6aca None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:10.312743 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ddf8d-4ad3-46b1-ab16-571b72ee3b39 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:10.324131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8d177-45e7-4252-9058-81f9cb081722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:10.325252 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8d177-45e7-4252-9058-81f9cb081722 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:10.325864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8d177-45e7-4252-9058-81f9cb081722 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:10.326435 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8d177-45e7-4252-9058-81f9cb081722 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:10.327290 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc8d177-45e7-4252-9058-81f9cb081722 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:10.328987 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc8d177-45e7-4252-9058-81f9cb081722 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:10.342920 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-8fc8d177-45e7-4252-9058-81f9cb081722 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 07 15:16:10.342920 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:10.342920 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 15:16:10.342920 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 15:16:10.342920 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 15:16:10.342920 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 15:16:10.342920 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 15:16:10.342920 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:16:10.342920 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 07 15:16:10.342920 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:10.342920 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 15:16:10.342920 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:10.342920 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:10.342920 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:10.342920 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:10.342920 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:10.342920 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:10.344014 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:10.344014 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:10.344014 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:10.344014 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:10.344014 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:10.344014 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:10.344014 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 15:16:10.344014 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 15:16:10.344014 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 15:16:10.344014 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 15:16:10.344014 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:10.344014 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:10.344014 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:10.344014 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:10.344014 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:10.344014 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:10.344866 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:10.344866 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:10.344866 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:10.344866 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:10.344866 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:10.344866 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:10.344866 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:10.344866 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:10.344866 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:10.344866 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:10.344866 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 15:16:10.344866 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:16:10.344866 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 07 15:16:10.344866 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:10.348087 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 428/878] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Jul 7 15:16:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:10.353733 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8d177-45e7-4252-9058-81f9cb081722 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:10.356991 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dea8fee-eb7f-4c92-8790-9982311d6107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:10.358065 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea8fee-eb7f-4c92-8790-9982311d6107 None admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:10.358347 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea8fee-eb7f-4c92-8790-9982311d6107 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:10.358620 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea8fee-eb7f-4c92-8790-9982311d6107 None admin] PATH_INFO: `/v3/services` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:10.360089 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dea8fee-eb7f-4c92-8790-9982311d6107 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:10.361094 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dea8fee-eb7f-4c92-8790-9982311d6107 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:10.434974 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:10.461893 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 451/879] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Jul 7 15:16:10 2022] POST /identity/v3/services => generated 269 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:16:10.463278 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8d177-45e7-4252-9058-81f9cb081722 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:10.512272 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94401f2a-ea7a-4b54-88e8-2d94a47fb32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:10.512872 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94401f2a-ea7a-4b54-88e8-2d94a47fb32d None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:10.513158 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94401f2a-ea7a-4b54-88e8-2d94a47fb32d None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:10.513428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94401f2a-ea7a-4b54-88e8-2d94a47fb32d None admin] PATH_INFO: `/v3/services` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:10.514209 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94401f2a-ea7a-4b54-88e8-2d94a47fb32d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:10.514575 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94401f2a-ea7a-4b54-88e8-2d94a47fb32d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:10.516212 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94401f2a-ea7a-4b54-88e8-2d94a47fb32d None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:10.522438 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 429/880] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Jul 7 15:16:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:10.528348 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dea8fee-eb7f-4c92-8790-9982311d6107 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:10.641992 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:10.646348 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dea8fee-eb7f-4c92-8790-9982311d6107 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:10.681900 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a105cd3-c1be-471c-bf53-2723cd2ddc1c None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:10.682207 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a105cd3-c1be-471c-bf53-2723cd2ddc1c None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:10.682479 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a105cd3-c1be-471c-bf53-2723cd2ddc1c None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:10.682781 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87008934-b506-4e82-9c49-d61e35ee8371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:10.683451 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87008934-b506-4e82-9c49-d61e35ee8371 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:10.683841 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87008934-b506-4e82-9c49-d61e35ee8371 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:10.684049 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87008934-b506-4e82-9c49-d61e35ee8371 None admin] PATH_INFO: `/v3/services` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:10.690061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 430/881] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:16:10.690172 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87008934-b506-4e82-9c49-d61e35ee8371 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:10.690172 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87008934-b506-4e82-9c49-d61e35ee8371 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:10.690172 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87008934-b506-4e82-9c49-d61e35ee8371 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:10.693451 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51ae33-c9a2-403c-9d17-9c21bbbcc0b4 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:10.693741 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51ae33-c9a2-403c-9d17-9c21bbbcc0b4 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:10.694018 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51ae33-c9a2-403c-9d17-9c21bbbcc0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:10.694415 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 452/882] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Jul 7 15:16:10 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:10.716921 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-ab51ae33-c9a2-403c-9d17-9c21bbbcc0b4 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:10.718158 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-ab51ae33-c9a2-403c-9d17-9c21bbbcc0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:10.805202 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 431/883] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:10 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:16:10.825457 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87008934-b506-4e82-9c49-d61e35ee8371 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:10.936046 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87008934-b506-4e82-9c49-d61e35ee8371 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:10.977033 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd94178-8584-4514-8adb-1cf269e6ccdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:10.978185 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd94178-8584-4514-8adb-1cf269e6ccdd None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:10.978688 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd94178-8584-4514-8adb-1cf269e6ccdd None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:10.979169 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd94178-8584-4514-8adb-1cf269e6ccdd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:10.991737 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd94178-8584-4514-8adb-1cf269e6ccdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:16:11.000411 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-efd94178-8584-4514-8adb-1cf269e6ccdd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 07 15:16:11.000411 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:11.000411 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:11.000411 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:11.000411 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:11.000411 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:11.000411 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:11.000411 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:11.000411 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:11.000411 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:11.000411 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:11.000411 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:11.000411 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:16:11.000411 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:16:11.000411 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:16:11.000411 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:16:11.001390 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:16:11.001390 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:16:11.001390 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:16:11.001390 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:16:11.001390 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:11.001390 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:11.001390 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:11.001390 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:11.001390 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:11.001390 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:11.001390 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:11.001390 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:11.001390 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:11.001390 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:11.001390 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:11.001390 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:11.002271 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:11.002271 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:11.002271 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:11.002271 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:11.002271 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:16:11.002271 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:16:11.002271 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:16:11.002271 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:16:11.002271 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:16:11.002271 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:16:11.002271 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 07 15:16:11.002271 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:11.009659 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 453/884] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Jul 7 15:16:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:11.011241 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51ae33-c9a2-403c-9d17-9c21bbbcc0b4 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:11.077093 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:11.116917 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51ae33-c9a2-403c-9d17-9c21bbbcc0b4 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:11.154678 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4edeaad-7ba7-445b-abc6-288f2e914a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:11.155345 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edeaad-7ba7-445b-abc6-288f2e914a46 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:11.155650 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edeaad-7ba7-445b-abc6-288f2e914a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:11.155925 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edeaad-7ba7-445b-abc6-288f2e914a46 None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:11.156671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4edeaad-7ba7-445b-abc6-288f2e914a46 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:11.156974 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4edeaad-7ba7-445b-abc6-288f2e914a46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:11.158324 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4edeaad-7ba7-445b-abc6-288f2e914a46 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:11.167069 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 432/885] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Jul 7 15:16:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:11.173445 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd94178-8584-4514-8adb-1cf269e6ccdd None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:11.203922 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:11.286004 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd94178-8584-4514-8adb-1cf269e6ccdd None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:11.320657 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4b3bb-65e9-4d02-8e43-c9aff6ad6f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:11.321818 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4b3bb-65e9-4d02-8e43-c9aff6ad6f53 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:11.322326 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4b3bb-65e9-4d02-8e43-c9aff6ad6f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:11.322801 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4b3bb-65e9-4d02-8e43-c9aff6ad6f53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:11.335176 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b4b3bb-65e9-4d02-8e43-c9aff6ad6f53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:11.336903 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b4b3bb-65e9-4d02-8e43-c9aff6ad6f53 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:11.355087 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-58b4b3bb-65e9-4d02-8e43-c9aff6ad6f53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:11.355087 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:11.355087 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:11.355087 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:11.355087 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:11.355087 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:11.355087 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:11.355087 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:11.355087 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:11.355087 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:11.355087 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:11.355087 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:11.355087 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:16:11.355087 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:16:11.355087 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:16:11.356064 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:16:11.356064 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:11.356064 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:11.356064 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:11.356064 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:11.356064 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:11.356064 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:11.356064 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:11.356064 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:11.356064 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:11.356064 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:11.356064 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:11.356064 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:11.356064 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:11.356064 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:11.356064 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:11.356064 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:11.357054 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:16:11.357054 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:16:11.357054 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:11.357054 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:11.363965 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 454/886] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:16:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:11.365208 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4edeaad-7ba7-445b-abc6-288f2e914a46 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:11.469690 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4edeaad-7ba7-445b-abc6-288f2e914a46 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:11.496271 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:11.505556 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf3da8-d10a-4ffe-b3d1-ee52119ad439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:11.510745 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf3da8-d10a-4ffe-b3d1-ee52119ad439 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:11.510745 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf3da8-d10a-4ffe-b3d1-ee52119ad439 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:11.510745 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf3da8-d10a-4ffe-b3d1-ee52119ad439 None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:11.510745 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbf3da8-d10a-4ffe-b3d1-ee52119ad439 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:11.510745 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbf3da8-d10a-4ffe-b3d1-ee52119ad439 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:11.510745 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbf3da8-d10a-4ffe-b3d1-ee52119ad439 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:11.524626 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dfc768-2e99-4d85-afd2-d47e53e778ed None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:11.525276 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dfc768-2e99-4d85-afd2-d47e53e778ed None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:11.525540 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 433/887] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:16:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:11.526019 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dfc768-2e99-4d85-afd2-d47e53e778ed None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:11.527603 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 455/888] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:16:11.531475 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf3da8-d10a-4ffe-b3d1-ee52119ad439 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:11.532908 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98ccf2-beb2-4d10-8b6c-ec60977e418d None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:11.533330 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98ccf2-beb2-4d10-8b6c-ec60977e418d None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:11.533819 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98ccf2-beb2-4d10-8b6c-ec60977e418d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:11.554916 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-eb98ccf2-beb2-4d10-8b6c-ec60977e418d None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:11.557009 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-eb98ccf2-beb2-4d10-8b6c-ec60977e418d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:11.638363 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf3da8-d10a-4ffe-b3d1-ee52119ad439 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:11.645800 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 456/889] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:11 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 07 15:16:11.668205 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb98ccf2-beb2-4d10-8b6c-ec60977e418d None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:11.671467 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:11.681492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc0db3-06a3-4964-8b82-0763bbad6c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:11.682301 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc0db3-06a3-4964-8b82-0763bbad6c09 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:11.682576 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc0db3-06a3-4964-8b82-0763bbad6c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:11.682901 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc0db3-06a3-4964-8b82-0763bbad6c09 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:11.690903 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc0db3-06a3-4964-8b82-0763bbad6c09 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:11.692315 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc0db3-06a3-4964-8b82-0763bbad6c09 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:11.713509 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-d2dc0db3-06a3-4964-8b82-0763bbad6c09 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 07 15:16:11.713509 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:11.713509 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:11.713509 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:11.713509 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:11.713509 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:11.713509 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:11.713509 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:11.713509 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:11.713509 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:11.713509 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:11.713509 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:11.713509 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 15:16:11.713509 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 15:16:11.713509 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 15:16:11.713509 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 15:16:11.714553 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:11.714553 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:11.714553 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 15:16:11.714553 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:16:11.714553 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 15:16:11.714553 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:16:11.714553 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:11.714553 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:11.714553 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:11.714553 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:11.714553 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:11.714553 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:11.714553 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:11.714553 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:11.714553 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:11.714553 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:11.714553 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:11.714553 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:11.715502 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:11.715502 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:11.715502 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 15:16:11.715502 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 15:16:11.715502 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 15:16:11.715502 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 15:16:11.715502 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 15:16:11.715502 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 15:16:11.715502 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 07 15:16:11.715502 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:11.719103 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 434/890] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Jul 7 15:16:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:11.722801 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc0db3-06a3-4964-8b82-0763bbad6c09 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:11.775451 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:11.794912 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb98ccf2-beb2-4d10-8b6c-ec60977e418d None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:11.842138 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc0db3-06a3-4964-8b82-0763bbad6c09 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:11.844486 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803fffa-0f27-4885-b80f-fb6a0676695c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:11.846405 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803fffa-0f27-4885-b80f-fb6a0676695c None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:11.846686 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803fffa-0f27-4885-b80f-fb6a0676695c None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:11.846960 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803fffa-0f27-4885-b80f-fb6a0676695c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:11.847935 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c803fffa-0f27-4885-b80f-fb6a0676695c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:11.849348 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c803fffa-0f27-4885-b80f-fb6a0676695c None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:11.865927 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-c803fffa-0f27-4885-b80f-fb6a0676695c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 07 15:16:11.865927 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:11.865927 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 15:16:11.865927 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 15:16:11.865927 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 15:16:11.865927 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 15:16:11.865927 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 15:16:11.865927 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:16:11.865927 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 07 15:16:11.865927 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:11.865927 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 15:16:11.865927 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:11.865927 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:11.865927 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:11.865927 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:11.865927 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:11.865927 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:11.867023 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:11.867023 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:11.867023 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:11.867023 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:11.867023 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:11.867023 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:11.867023 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 15:16:11.867023 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 15:16:11.867023 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 15:16:11.867023 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 15:16:11.867023 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:11.867023 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:11.867023 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:11.867023 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:11.867023 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:11.867023 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:11.867894 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:11.867894 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:11.867894 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:11.867894 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:11.867894 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:11.867894 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:11.867894 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:11.867894 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:11.867894 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:11.867894 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:11.867894 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 15:16:11.867894 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:16:11.867894 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 07 15:16:11.867894 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:11.868664 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 457/891] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Jul 7 15:16:11 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:16:11.875939 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803fffa-0f27-4885-b80f-fb6a0676695c None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:11.886415 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e8f97-29b2-4b2d-8ba9-035e9fa421a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:11.887103 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e8f97-29b2-4b2d-8ba9-035e9fa421a1 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:11.887389 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e8f97-29b2-4b2d-8ba9-035e9fa421a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:11.887705 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e8f97-29b2-4b2d-8ba9-035e9fa421a1 None admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:11.889317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505e8f97-29b2-4b2d-8ba9-035e9fa421a1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:11.889631 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505e8f97-29b2-4b2d-8ba9-035e9fa421a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:11.892006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505e8f97-29b2-4b2d-8ba9-035e9fa421a1 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:11.915385 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 435/892] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Jul 7 15:16:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:11.917173 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e8f97-29b2-4b2d-8ba9-035e9fa421a1 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:11.950404 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:11.987151 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803fffa-0f27-4885-b80f-fb6a0676695c None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:12.023295 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131a5d7-a344-410c-92cb-b810e4676916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:12.024320 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131a5d7-a344-410c-92cb-b810e4676916 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:12.024782 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e8f97-29b2-4b2d-8ba9-035e9fa421a1 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:12.025018 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131a5d7-a344-410c-92cb-b810e4676916 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:12.025492 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131a5d7-a344-410c-92cb-b810e4676916 None admin] PATH_INFO: `/v3/services` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:12.026954 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a131a5d7-a344-410c-92cb-b810e4676916 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:12.027141 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:12.028336 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a131a5d7-a344-410c-92cb-b810e4676916 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:12.029439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a131a5d7-a344-410c-92cb-b810e4676916 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:12.039248 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 458/893] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Jul 7 15:16:11 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:12.040283 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131a5d7-a344-410c-92cb-b810e4676916 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:12.066461 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e7aadf-4c32-4a28-9987-22e764e1db62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:12.067414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e7aadf-4c32-4a28-9987-22e764e1db62 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:12.067898 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e7aadf-4c32-4a28-9987-22e764e1db62 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:12.068321 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e7aadf-4c32-4a28-9987-22e764e1db62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:12.141251 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e7aadf-4c32-4a28-9987-22e764e1db62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:12.143342 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e7aadf-4c32-4a28-9987-22e764e1db62 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:12.149122 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131a5d7-a344-410c-92cb-b810e4676916 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:12.179738 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:12.186564 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7980a385-b9e9-4b5b-ad41-4fc5d87b7a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:12.187474 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7980a385-b9e9-4b5b-ad41-4fc5d87b7a5b None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:12.187986 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7980a385-b9e9-4b5b-ad41-4fc5d87b7a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:12.188487 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7980a385-b9e9-4b5b-ad41-4fc5d87b7a5b None admin] PATH_INFO: `/v3/services` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:12.189404 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7980a385-b9e9-4b5b-ad41-4fc5d87b7a5b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:12.189942 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7980a385-b9e9-4b5b-ad41-4fc5d87b7a5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:12.191503 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7980a385-b9e9-4b5b-ad41-4fc5d87b7a5b None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:12.201566 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 459/894] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Jul 7 15:16:12 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:12.205746 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7980a385-b9e9-4b5b-ad41-4fc5d87b7a5b None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:12.256496 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 436/895] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:16:11 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 07 15:16:12.257882 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e7aadf-4c32-4a28-9987-22e764e1db62 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:12.260772 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:12.308551 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7980a385-b9e9-4b5b-ad41-4fc5d87b7a5b None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:12.344780 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a079a7-38d5-4ccd-ae72-758e450dfbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:12.345530 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a079a7-38d5-4ccd-ae72-758e450dfbbc None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:12.345882 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a079a7-38d5-4ccd-ae72-758e450dfbbc None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:12.346226 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a079a7-38d5-4ccd-ae72-758e450dfbbc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:12.347085 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a079a7-38d5-4ccd-ae72-758e450dfbbc None admin] RBAC: Adding query filter params (interface=public, service_id=7e4734f17563422f88a91e377db1daa0, region_id=RegionOne) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:12.347465 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a079a7-38d5-4ccd-ae72-758e450dfbbc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:12.349316 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a079a7-38d5-4ccd-ae72-758e450dfbbc None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:12.354782 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 460/896] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Jul 7 15:16:12 2022] GET /identity/v3/endpoints?service_id=7e4734f17563422f88a91e377db1daa0&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:12.360696 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a079a7-38d5-4ccd-ae72-758e450dfbbc None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:12.365418 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e7aadf-4c32-4a28-9987-22e764e1db62 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:12.402477 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406eaa57-6b22-4b24-913e-bf0115cc9851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:12.403249 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406eaa57-6b22-4b24-913e-bf0115cc9851 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:12.403533 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406eaa57-6b22-4b24-913e-bf0115cc9851 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:12.403837 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406eaa57-6b22-4b24-913e-bf0115cc9851 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:12.415362 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406eaa57-6b22-4b24-913e-bf0115cc9851 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:12.416917 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406eaa57-6b22-4b24-913e-bf0115cc9851 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:12.424876 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:12.436334 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-406eaa57-6b22-4b24-913e-bf0115cc9851 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:12.436334 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:12.436334 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:12.436334 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:12.436334 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:12.436334 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:12.436334 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:12.436334 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:12.436334 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:12.436334 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:12.436334 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:12.436334 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:12.436334 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:16:12.436334 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:16:12.436334 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:16:12.437268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:16:12.437268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:12.437268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:12.437268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:12.437268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:12.437268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:12.437268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:12.437268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:12.437268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:12.437268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:12.437268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:12.437268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:12.437268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:12.437268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:12.437268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:12.437268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:12.437268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:12.438249 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:16:12.438249 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:16:12.438249 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:12.438249 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:12.442334 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 437/897] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:16:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:16:12.443684 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406eaa57-6b22-4b24-913e-bf0115cc9851 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:12.472933 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a079a7-38d5-4ccd-ae72-758e450dfbbc None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:12.507320 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:12.508939 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d1a07-5dfc-422e-89ce-a1bc8085875c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:12.509808 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d1a07-5dfc-422e-89ce-a1bc8085875c None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:12.510294 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d1a07-5dfc-422e-89ce-a1bc8085875c None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:12.510750 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d1a07-5dfc-422e-89ce-a1bc8085875c None admin] PATH_INFO: `/v3/services` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:12.511629 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4d1a07-5dfc-422e-89ce-a1bc8085875c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:12.513180 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4d1a07-5dfc-422e-89ce-a1bc8085875c None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:12.526254 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 461/898] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Jul 7 15:16:12 2022] GET /identity/v3/services => generated 1268 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:16:12.560110 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406eaa57-6b22-4b24-913e-bf0115cc9851 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:12.600881 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b593e1-3f06-4d4f-892d-33cf9d7866ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:12.601559 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b593e1-3f06-4d4f-892d-33cf9d7866ac None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:12.601846 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b593e1-3f06-4d4f-892d-33cf9d7866ac None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:12.602127 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b593e1-3f06-4d4f-892d-33cf9d7866ac None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:12.602872 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b593e1-3f06-4d4f-892d-33cf9d7866ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:12.603186 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b593e1-3f06-4d4f-892d-33cf9d7866ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:12.604624 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b593e1-3f06-4d4f-892d-33cf9d7866ac None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:12.623445 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 438/899] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:16:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:12.625364 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d1a07-5dfc-422e-89ce-a1bc8085875c None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:12.730147 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85820ab0-30aa-4b25-9e8f-008508667369 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:12.730443 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85820ab0-30aa-4b25-9e8f-008508667369 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:12.730625 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85820ab0-30aa-4b25-9e8f-008508667369 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:12.732854 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 439/900] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:16:12.738072 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2327fe4-adf6-4f7c-a2ba-f2aded2315ab None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:12.738358 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2327fe4-adf6-4f7c-a2ba-f2aded2315ab None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:12.738634 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2327fe4-adf6-4f7c-a2ba-f2aded2315ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:12.741257 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d1a07-5dfc-422e-89ce-a1bc8085875c None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:12.756139 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-e2327fe4-adf6-4f7c-a2ba-f2aded2315ab None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:12.757346 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-e2327fe4-adf6-4f7c-a2ba-f2aded2315ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:12.772992 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:12.777974 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf73a3d8-b77c-4682-99d0-fe5629f7c479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:12.778893 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73a3d8-b77c-4682-99d0-fe5629f7c479 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:12.779374 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73a3d8-b77c-4682-99d0-fe5629f7c479 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:12.779763 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73a3d8-b77c-4682-99d0-fe5629f7c479 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:12.786912 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf73a3d8-b77c-4682-99d0-fe5629f7c479 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:12.796074 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf73a3d8-b77c-4682-99d0-fe5629f7c479 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:12.804151 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-bf73a3d8-b77c-4682-99d0-fe5629f7c479 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:16:12.804151 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:12.804151 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:12.804151 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:12.804151 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:12.804151 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:12.804151 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:12.804151 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:12.804151 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:12.804151 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:12.804151 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:12.804151 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:12.804151 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 15:16:12.804151 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 15:16:12.804151 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 15:16:12.805055 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 15:16:12.805055 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:12.805055 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:12.805055 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:12.805055 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:12.805055 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:12.805055 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:12.805055 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:12.805055 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:12.805055 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:12.805055 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:12.805055 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:12.805055 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:12.805055 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:12.805055 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:12.805055 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:12.805055 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:12.805967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 15:16:12.805967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 15:16:12.805967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 15:16:12.805967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 15:16:12.805967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 15:16:12.805967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 15:16:12.805967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:16:12.805967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:12.805967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 462/901] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:16:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:12.806910 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c52cf7-5e7d-4678-8566-218ff636e2a4 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:12.807172 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c52cf7-5e7d-4678-8566-218ff636e2a4 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:12.807443 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c52cf7-5e7d-4678-8566-218ff636e2a4 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:12.811146 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 463/902] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:16:12.812138 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c52cf7-5e7d-4678-8566-218ff636e2a4 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:12.839727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 440/903] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:12 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 07 15:16:12.840943 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0766fc0-a176-4ab1-a5d2-2fa715e2b3de None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:12.841232 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0766fc0-a176-4ab1-a5d2-2fa715e2b3de None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:12.841539 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0766fc0-a176-4ab1-a5d2-2fa715e2b3de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:12.860534 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-e0766fc0-a176-4ab1-a5d2-2fa715e2b3de None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:12.861714 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-e0766fc0-a176-4ab1-a5d2-2fa715e2b3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:12.906635 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:12.932160 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c52cf7-5e7d-4678-8566-218ff636e2a4 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:12.932829 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 441/904] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:12 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:16:12.933671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0766fc0-a176-4ab1-a5d2-2fa715e2b3de None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:12.977278 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7fe607-db20-4a28-888e-57b165149611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:12.977942 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fe607-db20-4a28-888e-57b165149611 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:12.978320 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fe607-db20-4a28-888e-57b165149611 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:12.978953 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fe607-db20-4a28-888e-57b165149611 None admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:12.980224 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7fe607-db20-4a28-888e-57b165149611 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:12.980490 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7fe607-db20-4a28-888e-57b165149611 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:12.982740 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7fe607-db20-4a28-888e-57b165149611 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:13.004272 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 464/905] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Jul 7 15:16:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:13.004272 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7fe607-db20-4a28-888e-57b165149611 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:13.038408 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0766fc0-a176-4ab1-a5d2-2fa715e2b3de None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:13.060202 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:13.073828 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87269372-4182-435d-9907-426eeaba6549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:13.074635 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87269372-4182-435d-9907-426eeaba6549 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:13.074919 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87269372-4182-435d-9907-426eeaba6549 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:13.075188 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87269372-4182-435d-9907-426eeaba6549 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:13.075917 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87269372-4182-435d-9907-426eeaba6549 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:13.077266 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87269372-4182-435d-9907-426eeaba6549 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:13.089414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-87269372-4182-435d-9907-426eeaba6549 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 07 15:16:13.089414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:13.089414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 15:16:13.089414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 15:16:13.089414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 15:16:13.089414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 15:16:13.089414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 15:16:13.089414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:16:13.089414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 07 15:16:13.089414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:13.089414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 15:16:13.089414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:13.089414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:13.089414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:13.089414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:13.089414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:13.090358 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:13.090358 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:13.090358 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:13.090358 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:13.090358 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:13.090358 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:13.090358 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:13.090358 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 15:16:13.090358 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 15:16:13.090358 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 15:16:13.090358 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 15:16:13.090358 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:13.090358 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:13.090358 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:13.090358 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:13.090358 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:13.096535 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:13.096535 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:13.096535 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:13.096535 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:13.096535 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:13.096535 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:13.096535 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:13.096535 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:13.096535 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:13.096535 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:13.096535 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:13.096535 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 15:16:13.096535 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:16:13.096535 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 07 15:16:13.096535 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:13.096535 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 442/906] 10.222.0.25 () {56 vars in 1132 bytes} [Thu Jul 7 15:16:12 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:13.097408 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87269372-4182-435d-9907-426eeaba6549 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:13.122275 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7fe607-db20-4a28-888e-57b165149611 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:13.143443 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:13.158318 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2dc078-6b1d-4712-a79d-b07b27d1326c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:13.158984 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2dc078-6b1d-4712-a79d-b07b27d1326c None admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:13.159269 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2dc078-6b1d-4712-a79d-b07b27d1326c None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:13.159553 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2dc078-6b1d-4712-a79d-b07b27d1326c None admin] PATH_INFO: `/v3/services` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:13.160296 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2dc078-6b1d-4712-a79d-b07b27d1326c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:13.161664 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2dc078-6b1d-4712-a79d-b07b27d1326c None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:13.208611 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87269372-4182-435d-9907-426eeaba6549 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:13.244771 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47075e84-4130-4f67-8772-3548cc133922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:13.245353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47075e84-4130-4f67-8772-3548cc133922 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:13.245637 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47075e84-4130-4f67-8772-3548cc133922 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:13.245909 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47075e84-4130-4f67-8772-3548cc133922 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:13.246739 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47075e84-4130-4f67-8772-3548cc133922 None admin] RBAC: Adding query filter params (role.id=d5899b9e6b9940aa9837652e51a23908, scope.project.id=c5260c7fd3614e9e93b167c86a94260e, user.id=cd3c04f39d774998b66e6ce920925691) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:13.247052 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47075e84-4130-4f67-8772-3548cc133922 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:13.248529 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47075e84-4130-4f67-8772-3548cc133922 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:13.254397 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 443/907] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:16:13 2022] GET /identity/v3/role_assignments?user.id=cd3c04f39d774998b66e6ce920925691&scope.project.id=c5260c7fd3614e9e93b167c86a94260e&role.id=d5899b9e6b9940aa9837652e51a23908 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:13.255205 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47075e84-4130-4f67-8772-3548cc133922 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:13.288539 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 465/908] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Jul 7 15:16:12 2022] POST /identity/v3/services => generated 241 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:16:13.407617 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47075e84-4130-4f67-8772-3548cc133922 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:13.410930 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:13.455148 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae7f02-98d4-41d1-a764-d670a937ec6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:13.456188 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae7f02-98d4-41d1-a764-d670a937ec6f None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:13.456672 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae7f02-98d4-41d1-a764-d670a937ec6f None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:13.457178 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae7f02-98d4-41d1-a764-d670a937ec6f None admin] PATH_INFO: `/v3/services` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:13.458131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ae7f02-98d4-41d1-a764-d670a937ec6f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:13.458645 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ae7f02-98d4-41d1-a764-d670a937ec6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:13.461156 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ae7f02-98d4-41d1-a764-d670a937ec6f None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:13.467932 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 444/909] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Jul 7 15:16:13 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:13.473527 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2dc078-6b1d-4712-a79d-b07b27d1326c None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:13.580809 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2dc078-6b1d-4712-a79d-b07b27d1326c None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:13.583601 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:13.623533 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ad0b5a-8702-45d1-9c3a-6e239bc1d4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:13.624240 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad0b5a-8702-45d1-9c3a-6e239bc1d4df None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:13.624582 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad0b5a-8702-45d1-9c3a-6e239bc1d4df None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:13.624867 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad0b5a-8702-45d1-9c3a-6e239bc1d4df None admin] PATH_INFO: `/v3/services` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:13.625585 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ad0b5a-8702-45d1-9c3a-6e239bc1d4df None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:13.626040 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ad0b5a-8702-45d1-9c3a-6e239bc1d4df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:13.627417 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ad0b5a-8702-45d1-9c3a-6e239bc1d4df None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:13.633146 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 466/910] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Jul 7 15:16:13 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:13.640839 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae7f02-98d4-41d1-a764-d670a937ec6f None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:13.749192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae7f02-98d4-41d1-a764-d670a937ec6f None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:13.785281 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84612f15-23b9-4468-990b-5f51b1656c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:13.786356 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84612f15-23b9-4468-990b-5f51b1656c92 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:13.786641 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84612f15-23b9-4468-990b-5f51b1656c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:13.786920 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84612f15-23b9-4468-990b-5f51b1656c92 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:13.787758 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84612f15-23b9-4468-990b-5f51b1656c92 None admin] RBAC: Adding query filter params (interface=public, service_id=6645002b92a34d97bae518a2f9f5177d, region_id=RegionOne) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:13.788075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84612f15-23b9-4468-990b-5f51b1656c92 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:13.789443 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84612f15-23b9-4468-990b-5f51b1656c92 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:13.794993 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 445/911] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Jul 7 15:16:13 2022] GET /identity/v3/endpoints?service_id=6645002b92a34d97bae518a2f9f5177d&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:13.800462 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ad0b5a-8702-45d1-9c3a-6e239bc1d4df None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:13.906122 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ad0b5a-8702-45d1-9c3a-6e239bc1d4df None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:13.940667 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d31af0-8f8e-4691-99ec-a7ae0c929764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:13.941340 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d31af0-8f8e-4691-99ec-a7ae0c929764 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:13.941625 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d31af0-8f8e-4691-99ec-a7ae0c929764 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:13.941905 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d31af0-8f8e-4691-99ec-a7ae0c929764 None admin] PATH_INFO: `/v3/services` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:13.942594 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d31af0-8f8e-4691-99ec-a7ae0c929764 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:13.944002 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d31af0-8f8e-4691-99ec-a7ae0c929764 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:13.953873 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 467/912] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Jul 7 15:16:13 2022] GET /identity/v3/services => generated 1497 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:16:14.603480 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe725fae-b3ba-431f-b43b-eeda30128a4e None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:14.604056 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe725fae-b3ba-431f-b43b-eeda30128a4e None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:14.604056 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe725fae-b3ba-431f-b43b-eeda30128a4e None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:14.605646 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 446/913] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:16:14.611510 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130bf59c-b4e5-42e9-b80e-424fb5a7c89f None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:14.611960 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130bf59c-b4e5-42e9-b80e-424fb5a7c89f None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:14.612139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130bf59c-b4e5-42e9-b80e-424fb5a7c89f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:14.629709 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-130bf59c-b4e5-42e9-b80e-424fb5a7c89f None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:14.631131 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-130bf59c-b4e5-42e9-b80e-424fb5a7c89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:14.664536 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:14.711454 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 468/914] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 07 15:16:14.736251 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe725fae-b3ba-431f-b43b-eeda30128a4e None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:14.765384 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:14.843044 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe725fae-b3ba-431f-b43b-eeda30128a4e None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:14.879830 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9470ce9d-fd04-45ab-a9d9-93e51493d0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:14.880557 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470ce9d-fd04-45ab-a9d9-93e51493d0fd None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:14.880841 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470ce9d-fd04-45ab-a9d9-93e51493d0fd None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:14.881119 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470ce9d-fd04-45ab-a9d9-93e51493d0fd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:14.881777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9470ce9d-fd04-45ab-a9d9-93e51493d0fd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:14.883138 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9470ce9d-fd04-45ab-a9d9-93e51493d0fd None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:14.896389 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-9470ce9d-fd04-45ab-a9d9-93e51493d0fd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 07 15:16:14.896389 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:14.896389 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 15:16:14.896389 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 15:16:14.896389 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 15:16:14.896389 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 15:16:14.896389 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 15:16:14.896389 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:16:14.896389 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 07 15:16:14.896389 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:14.896389 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 15:16:14.896389 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:14.896389 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:14.896389 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:14.896389 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:14.896389 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:14.896389 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:14.897476 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:14.897476 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:14.897476 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:14.897476 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:14.897476 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:14.897476 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:14.897476 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 15:16:14.897476 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 15:16:14.897476 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 15:16:14.897476 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 15:16:14.897476 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:14.897476 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:14.897476 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:14.897476 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:14.897476 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:14.897476 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:14.898391 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:14.898391 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:14.898391 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:14.898391 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:14.898391 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:14.898391 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:14.898391 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:14.898391 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:14.898391 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:14.898391 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:14.898391 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 15:16:14.898391 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:16:14.898391 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 07 15:16:14.898391 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:14.898391 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 447/915] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Jul 7 15:16:14 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:14.904468 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130bf59c-b4e5-42e9-b80e-424fb5a7c89f None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:15.018454 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130bf59c-b4e5-42e9-b80e-424fb5a7c89f None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:15.054285 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c755a59-9a77-40c8-93ea-be4240dc008d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:15.054956 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c755a59-9a77-40c8-93ea-be4240dc008d None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:15.055241 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c755a59-9a77-40c8-93ea-be4240dc008d None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:15.055517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c755a59-9a77-40c8-93ea-be4240dc008d None admin] PATH_INFO: `/v3/services` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:15.056261 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c755a59-9a77-40c8-93ea-be4240dc008d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:15.056569 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c755a59-9a77-40c8-93ea-be4240dc008d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:15.057939 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c755a59-9a77-40c8-93ea-be4240dc008d None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:15.063433 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 469/916] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Jul 7 15:16:14 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:15.068982 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9470ce9d-fd04-45ab-a9d9-93e51493d0fd None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:15.169176 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:15.177942 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9470ce9d-fd04-45ab-a9d9-93e51493d0fd None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:15.213683 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23299914-7d41-4953-b80b-c117839db1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:15.214352 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23299914-7d41-4953-b80b-c117839db1a5 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:15.214636 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23299914-7d41-4953-b80b-c117839db1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:15.214910 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23299914-7d41-4953-b80b-c117839db1a5 None admin] PATH_INFO: `/v3/services` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:15.215655 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23299914-7d41-4953-b80b-c117839db1a5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:15.216182 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23299914-7d41-4953-b80b-c117839db1a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:15.218284 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23299914-7d41-4953-b80b-c117839db1a5 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:15.224367 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 448/917] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Jul 7 15:16:15 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:15.230085 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c755a59-9a77-40c8-93ea-be4240dc008d None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:15.260952 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:15.351097 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c755a59-9a77-40c8-93ea-be4240dc008d None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:15.389431 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd24e7-7ff5-42ad-9841-de7da6f93528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:15.390402 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd24e7-7ff5-42ad-9841-de7da6f93528 None admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:15.390871 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd24e7-7ff5-42ad-9841-de7da6f93528 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:15.391349 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd24e7-7ff5-42ad-9841-de7da6f93528 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:15.392310 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bd24e7-7ff5-42ad-9841-de7da6f93528 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:15.393874 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bd24e7-7ff5-42ad-9841-de7da6f93528 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:15.461398 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68505330-a034-4117-a415-ea31f2f24b6c None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:15.461653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68505330-a034-4117-a415-ea31f2f24b6c None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:15.462074 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68505330-a034-4117-a415-ea31f2f24b6c None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:15.463692 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 449/918] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:16:15.468383 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c75cc-333c-46f2-b541-5ad814a3d6c6 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:15.468672 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c75cc-333c-46f2-b541-5ad814a3d6c6 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:15.468952 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c75cc-333c-46f2-b541-5ad814a3d6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:15.474576 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 470/919] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Jul 7 15:16:15 2022] POST /identity/v3/endpoints => generated 346 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:16:15.487837 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-d03c75cc-333c-46f2-b541-5ad814a3d6c6 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:15.489103 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-d03c75cc-333c-46f2-b541-5ad814a3d6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:15.537454 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9cf9b-4b43-4245-91df-0552fc3b90b3 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:15.538127 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9cf9b-4b43-4245-91df-0552fc3b90b3 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:15.538632 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9cf9b-4b43-4245-91df-0552fc3b90b3 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:15.540269 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 471/920] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:16:15.546403 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a54ae4-ae0d-4a38-b102-e33956da53ba None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:15.547035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a54ae4-ae0d-4a38-b102-e33956da53ba None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:15.547514 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a54ae4-ae0d-4a38-b102-e33956da53ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:15.567600 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-e9a54ae4-ae0d-4a38-b102-e33956da53ba None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:15.569178 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-e9a54ae4-ae0d-4a38-b102-e33956da53ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:15.577196 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 450/921] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 07 15:16:15.599830 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c75cc-333c-46f2-b541-5ad814a3d6c6 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:15.679679 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 472/922] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 07 15:16:15.697689 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a54ae4-ae0d-4a38-b102-e33956da53ba None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:15.708244 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:15.715718 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:15.732558 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c75cc-333c-46f2-b541-5ad814a3d6c6 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:15.777227 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e2368d-36a6-45ab-96e9-236fba0b28ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:15.778132 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2368d-36a6-45ab-96e9-236fba0b28ae None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:15.778338 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2368d-36a6-45ab-96e9-236fba0b28ae None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:15.778609 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2368d-36a6-45ab-96e9-236fba0b28ae None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:15.789812 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e2368d-36a6-45ab-96e9-236fba0b28ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:16:15.798777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-13e2368d-36a6-45ab-96e9-236fba0b28ae None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 07 15:16:15.798777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:15.798777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:15.798777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:15.798777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:15.798777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:15.798777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:15.798777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:15.798777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:15.798777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:15.798777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:15.798777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:15.798777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:16:15.798777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:16:15.798777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:16:15.798777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:16:15.799821 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:16:15.799821 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:16:15.799821 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:16:15.799821 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:16:15.799821 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:15.799821 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:15.799821 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:15.799821 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:15.799821 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:15.799821 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:15.799821 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:15.799821 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:15.799821 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:15.799821 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:15.799821 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:15.799821 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:15.801006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:15.801006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:15.801006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:15.801006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:15.801006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:16:15.801006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:16:15.801006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:16:15.801006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:16:15.801006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:16:15.801006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:16:15.801006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 07 15:16:15.801006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:15.801006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 451/923] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Jul 7 15:16:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:16:15.806169 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e2368d-36a6-45ab-96e9-236fba0b28ae None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:15.830318 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a54ae4-ae0d-4a38-b102-e33956da53ba None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:15.876379 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f53b1f-0129-4608-8e42-69c4056f31fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:15.878646 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f53b1f-0129-4608-8e42-69c4056f31fa None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:15.878646 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f53b1f-0129-4608-8e42-69c4056f31fa None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:15.878646 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f53b1f-0129-4608-8e42-69c4056f31fa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:15.879946 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f53b1f-0129-4608-8e42-69c4056f31fa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:15.881795 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f53b1f-0129-4608-8e42-69c4056f31fa None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:15.898249 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-04f53b1f-0129-4608-8e42-69c4056f31fa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 07 15:16:15.898249 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:15.898249 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 15:16:15.898249 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 15:16:15.898249 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 15:16:15.898249 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 15:16:15.898249 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 15:16:15.898249 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:16:15.898249 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 07 15:16:15.898249 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:15.898249 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 15:16:15.898249 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:15.898249 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:15.898249 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:15.898249 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:15.898249 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:15.898249 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:15.899596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:15.899596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:15.899596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:15.899596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:15.899596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:15.899596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:15.899596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 15:16:15.899596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 15:16:15.899596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 15:16:15.899596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 15:16:15.899596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:15.899596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:15.899596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:15.899596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:15.899596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:15.899596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:15.900637 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:15.900637 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:15.900637 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:15.900637 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:15.900637 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:15.900637 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:15.900637 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:15.900637 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:15.900637 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:15.900637 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:15.900637 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 15:16:15.900637 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:16:15.900637 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 07 15:16:15.900637 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:15.903890 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 473/924] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Jul 7 15:16:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:15.909909 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f53b1f-0129-4608-8e42-69c4056f31fa None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:15.925157 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e2368d-36a6-45ab-96e9-236fba0b28ae None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:15.971900 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732372c-fdda-4d2e-ba4c-7f8a2e1e6f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:15.973031 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732372c-fdda-4d2e-ba4c-7f8a2e1e6f52 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:15.973455 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732372c-fdda-4d2e-ba4c-7f8a2e1e6f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:15.973884 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732372c-fdda-4d2e-ba4c-7f8a2e1e6f52 None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:15.974770 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d732372c-fdda-4d2e-ba4c-7f8a2e1e6f52 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:15.975234 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d732372c-fdda-4d2e-ba4c-7f8a2e1e6f52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:15.976759 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d732372c-fdda-4d2e-ba4c-7f8a2e1e6f52 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:15.986567 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 452/925] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Jul 7 15:16:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:15.992193 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732372c-fdda-4d2e-ba4c-7f8a2e1e6f52 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:15.996195 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:16.014861 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:16.038687 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f53b1f-0129-4608-8e42-69c4056f31fa None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:16.075438 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e102707f-724e-4d54-a7dc-7199f5b48943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:16.076166 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102707f-724e-4d54-a7dc-7199f5b48943 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:16.076445 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102707f-724e-4d54-a7dc-7199f5b48943 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:16.076715 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102707f-724e-4d54-a7dc-7199f5b48943 None admin] PATH_INFO: `/v3/services` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:16.077416 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e102707f-724e-4d54-a7dc-7199f5b48943 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:16.077721 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e102707f-724e-4d54-a7dc-7199f5b48943 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:16.079104 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e102707f-724e-4d54-a7dc-7199f5b48943 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:16.088377 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 474/926] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Jul 7 15:16:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:16.088377 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e8ab1-269e-4c20-8565-cb702a5a5ead None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:16.088377 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e8ab1-269e-4c20-8565-cb702a5a5ead None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:16.088377 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e8ab1-269e-4c20-8565-cb702a5a5ead None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:16.088377 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 475/927] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:16:16.092346 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e8ab1-269e-4c20-8565-cb702a5a5ead None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:16.110473 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732372c-fdda-4d2e-ba4c-7f8a2e1e6f52 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:16.148882 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c9a4c-b68b-45e0-84ea-44c5bd377adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:16.149716 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c9a4c-b68b-45e0-84ea-44c5bd377adb None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:16.149946 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c9a4c-b68b-45e0-84ea-44c5bd377adb None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:16.150220 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c9a4c-b68b-45e0-84ea-44c5bd377adb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:16.162153 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2c9a4c-b68b-45e0-84ea-44c5bd377adb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:16.163172 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2c9a4c-b68b-45e0-84ea-44c5bd377adb None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:16.180913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-cf2c9a4c-b68b-45e0-84ea-44c5bd377adb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:16.180913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:16.180913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:16.180913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:16.180913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:16.180913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:16.180913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:16.180913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:16.180913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:16.180913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:16.180913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:16.180913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:16.180913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:16:16.180913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:16:16.180913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:16:16.181848 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:16:16.181848 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:16.181848 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:16.181848 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:16.181848 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:16.181848 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:16.181848 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:16.181848 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:16.181848 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:16.181848 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:16.181848 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:16.181848 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:16.181848 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:16.181848 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:16.181848 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:16.181848 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:16.181848 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:16.182757 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:16:16.182757 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:16:16.182757 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:16.182757 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:16.182757 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 453/928] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:16:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:16.183455 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9b4a2-28b8-43a8-bff9-46ff49b7cf8d None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:16.183661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9b4a2-28b8-43a8-bff9-46ff49b7cf8d None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:16.183827 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9b4a2-28b8-43a8-bff9-46ff49b7cf8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:16.201250 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-eac9b4a2-28b8-43a8-bff9-46ff49b7cf8d None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:16.202442 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-eac9b4a2-28b8-43a8-bff9-46ff49b7cf8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:16.203291 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e8ab1-269e-4c20-8565-cb702a5a5ead None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:16.238594 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a187b-1d3f-427a-a078-8447f03d3645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:16.239172 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a187b-1d3f-427a-a078-8447f03d3645 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:16.239452 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a187b-1d3f-427a-a078-8447f03d3645 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:16.239756 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a187b-1d3f-427a-a078-8447f03d3645 None admin] PATH_INFO: `/v3/services` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:16.240481 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351a187b-1d3f-427a-a078-8447f03d3645 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:16.240784 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351a187b-1d3f-427a-a078-8447f03d3645 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:16.242138 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351a187b-1d3f-427a-a078-8447f03d3645 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:16.251807 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 476/929] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Jul 7 15:16:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:16.251807 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a187b-1d3f-427a-a078-8447f03d3645 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:16.255251 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:16.259905 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:16.276903 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 454/930] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:16:16.277546 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac9b4a2-28b8-43a8-bff9-46ff49b7cf8d None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:16.364088 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a187b-1d3f-427a-a078-8447f03d3645 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:16.389579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac9b4a2-28b8-43a8-bff9-46ff49b7cf8d None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:16.397565 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14560f65-3537-408d-95b8-ea500fb19505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:16.398214 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14560f65-3537-408d-95b8-ea500fb19505 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:16.398492 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14560f65-3537-408d-95b8-ea500fb19505 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:16.398764 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14560f65-3537-408d-95b8-ea500fb19505 None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:16.399475 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14560f65-3537-408d-95b8-ea500fb19505 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:16.399832 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14560f65-3537-408d-95b8-ea500fb19505 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:16.401184 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14560f65-3537-408d-95b8-ea500fb19505 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:16.412447 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 477/931] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:16.413209 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14560f65-3537-408d-95b8-ea500fb19505 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:16.432703 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc67d8-4bae-46ab-a222-2f05ce8bb54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:16.433684 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc67d8-4bae-46ab-a222-2f05ce8bb54b None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:16.434163 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc67d8-4bae-46ab-a222-2f05ce8bb54b None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:16.434619 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc67d8-4bae-46ab-a222-2f05ce8bb54b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:16.436277 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfc67d8-4bae-46ab-a222-2f05ce8bb54b None admin] RBAC: Adding query filter params (interface=public, service_id=2ce7dc3ac1a44b938f915b5c261b4c6e, region_id=RegionOne) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:16.436781 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfc67d8-4bae-46ab-a222-2f05ce8bb54b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:16.438329 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfc67d8-4bae-46ab-a222-2f05ce8bb54b None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:16.448265 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 455/932] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Jul 7 15:16:16 2022] GET /identity/v3/endpoints?service_id=2ce7dc3ac1a44b938f915b5c261b4c6e&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:16.448265 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc67d8-4bae-46ab-a222-2f05ce8bb54b None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:16.481768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:16.498216 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:16.520503 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14560f65-3537-408d-95b8-ea500fb19505 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:16.550081 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc67d8-4bae-46ab-a222-2f05ce8bb54b None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:16.556360 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbb1ced-1504-4e92-b10e-5ea4fbe8a6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:16.557176 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb1ced-1504-4e92-b10e-5ea4fbe8a6d5 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:16.557460 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb1ced-1504-4e92-b10e-5ea4fbe8a6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:16.557737 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb1ced-1504-4e92-b10e-5ea4fbe8a6d5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:16.558396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbb1ced-1504-4e92-b10e-5ea4fbe8a6d5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:16.560759 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbb1ced-1504-4e92-b10e-5ea4fbe8a6d5 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:16.573745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-8cbb1ced-1504-4e92-b10e-5ea4fbe8a6d5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 07 15:16:16.573745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:16.573745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 15:16:16.573745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 15:16:16.573745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 15:16:16.573745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 15:16:16.573745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 15:16:16.573745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:16:16.573745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 07 15:16:16.573745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:16.573745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 15:16:16.573745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:16.573745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:16.573745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:16.573745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:16.573745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:16.574708 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:16.574708 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:16.574708 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:16.574708 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:16.574708 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:16.574708 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:16.574708 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:16.574708 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 15:16:16.574708 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 15:16:16.574708 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 15:16:16.574708 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 15:16:16.574708 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:16.574708 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:16.574708 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:16.574708 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:16.574708 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:16.575602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:16.575602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:16.575602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:16.575602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:16.575602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:16.575602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:16.575602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:16.575602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:16.575602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:16.575602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:16.575602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:16.575602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 15:16:16.575602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:16:16.575602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 07 15:16:16.575602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:16.575602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 478/933] 10.222.0.25 () {56 vars in 1132 bytes} [Thu Jul 7 15:16:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:16.576496 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbb1ced-1504-4e92-b10e-5ea4fbe8a6d5 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:16.593764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf4ae40-15fc-4385-b7b9-8a3adf2a3376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:16.594577 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf4ae40-15fc-4385-b7b9-8a3adf2a3376 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:16.594855 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf4ae40-15fc-4385-b7b9-8a3adf2a3376 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:16.595123 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf4ae40-15fc-4385-b7b9-8a3adf2a3376 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:16.603471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf4ae40-15fc-4385-b7b9-8a3adf2a3376 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:16.604940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf4ae40-15fc-4385-b7b9-8a3adf2a3376 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:16.621208 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-ddf4ae40-15fc-4385-b7b9-8a3adf2a3376 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 07 15:16:16.621208 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:16.621208 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:16.621208 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:16.621208 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:16.621208 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:16.621208 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:16.621208 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:16.621208 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:16.621208 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:16.621208 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:16.621208 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:16.621208 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 15:16:16.621208 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 15:16:16.621208 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 15:16:16.621208 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 15:16:16.622245 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:16.622245 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:16.622245 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 15:16:16.622245 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:16:16.622245 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 15:16:16.622245 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:16:16.622245 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:16.622245 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:16.622245 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:16.622245 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:16.622245 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:16.622245 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:16.622245 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:16.622245 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:16.622245 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:16.622245 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:16.622245 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:16.622245 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:16.623871 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:16.623871 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:16.623871 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 15:16:16.623871 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 15:16:16.623871 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 15:16:16.623871 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 15:16:16.623871 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 15:16:16.623871 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 15:16:16.623871 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 07 15:16:16.623871 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:16.623871 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 456/934] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Jul 7 15:16:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:16.623871 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf4ae40-15fc-4385-b7b9-8a3adf2a3376 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:16.686428 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbb1ced-1504-4e92-b10e-5ea4fbe8a6d5 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:16.714873 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:16.724701 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc15afc-4554-4a8a-9f6a-343b63b67f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:16.725372 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc15afc-4554-4a8a-9f6a-343b63b67f22 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:16.725661 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc15afc-4554-4a8a-9f6a-343b63b67f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:16.725938 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc15afc-4554-4a8a-9f6a-343b63b67f22 None admin] PATH_INFO: `/v3/services` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:16.726629 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc15afc-4554-4a8a-9f6a-343b63b67f22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:16.728040 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc15afc-4554-4a8a-9f6a-343b63b67f22 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:16.734027 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 479/935] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Jul 7 15:16:16 2022] GET /identity/v3/services => generated 1497 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:16:16.734805 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc15afc-4554-4a8a-9f6a-343b63b67f22 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:16.735411 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf4ae40-15fc-4385-b7b9-8a3adf2a3376 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:16.756053 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:16.793008 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effec75a-d4eb-44a5-9ed7-c4f525d07d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:16.794015 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effec75a-d4eb-44a5-9ed7-c4f525d07d06 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:16.795045 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effec75a-d4eb-44a5-9ed7-c4f525d07d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:16.795581 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effec75a-d4eb-44a5-9ed7-c4f525d07d06 None admin] PATH_INFO: `/v3/services` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:16.796997 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effec75a-d4eb-44a5-9ed7-c4f525d07d06 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:16.797815 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effec75a-d4eb-44a5-9ed7-c4f525d07d06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:16.799774 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effec75a-d4eb-44a5-9ed7-c4f525d07d06 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:16.807685 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 457/936] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Jul 7 15:16:16 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:16.812673 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effec75a-d4eb-44a5-9ed7-c4f525d07d06 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:16.865663 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc15afc-4554-4a8a-9f6a-343b63b67f22 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:16.907265 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca3aee-f9c7-4a17-b95d-fac05f998962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:16.907974 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca3aee-f9c7-4a17-b95d-fac05f998962 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:16.908254 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca3aee-f9c7-4a17-b95d-fac05f998962 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:16.908696 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca3aee-f9c7-4a17-b95d-fac05f998962 None admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:16.909542 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ca3aee-f9c7-4a17-b95d-fac05f998962 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:16.909849 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ca3aee-f9c7-4a17-b95d-fac05f998962 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:16.911263 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ca3aee-f9c7-4a17-b95d-fac05f998962 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:16.935917 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 480/937] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Jul 7 15:16:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:16.941924 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca3aee-f9c7-4a17-b95d-fac05f998962 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:16.948925 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effec75a-d4eb-44a5-9ed7-c4f525d07d06 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:16.998849 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb54ec-8e25-4670-a193-d5b86273d0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:16.999523 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb54ec-8e25-4670-a193-d5b86273d0cb None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:16.999851 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb54ec-8e25-4670-a193-d5b86273d0cb None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:17.000129 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb54ec-8e25-4670-a193-d5b86273d0cb None admin] PATH_INFO: `/v3/services` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:17.000847 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bb54ec-8e25-4670-a193-d5b86273d0cb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:17.001157 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bb54ec-8e25-4670-a193-d5b86273d0cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:17.002525 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bb54ec-8e25-4670-a193-d5b86273d0cb None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:17.011680 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 458/938] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Jul 7 15:16:16 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:17.015600 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb54ec-8e25-4670-a193-d5b86273d0cb None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:17.019469 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:17.061337 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:17.082798 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca3aee-f9c7-4a17-b95d-fac05f998962 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:17.129509 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb54ec-8e25-4670-a193-d5b86273d0cb None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:17.133578 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee51d5-ecef-422e-926a-33823e04b423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:17.134595 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee51d5-ecef-422e-926a-33823e04b423 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:17.135029 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee51d5-ecef-422e-926a-33823e04b423 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:17.135469 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee51d5-ecef-422e-926a-33823e04b423 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:17.149805 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcee51d5-ecef-422e-926a-33823e04b423 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:17.151495 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcee51d5-ecef-422e-926a-33823e04b423 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:17.166370 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5ce40-9856-442e-a285-694ad5c5d80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:17.167135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5ce40-9856-442e-a285-694ad5c5d80a None admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:17.167511 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5ce40-9856-442e-a285-694ad5c5d80a None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:17.167906 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5ce40-9856-442e-a285-694ad5c5d80a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:17.168789 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b5ce40-9856-442e-a285-694ad5c5d80a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:17.170254 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b5ce40-9856-442e-a285-694ad5c5d80a None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:17.174268 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-bcee51d5-ecef-422e-926a-33823e04b423 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:17.174268 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:17.174268 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:17.174268 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:17.174268 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:17.174268 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:17.174268 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:17.174268 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:17.174268 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:17.174268 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:17.174268 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:17.174268 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:17.174268 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:16:17.174268 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:16:17.174268 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:16:17.175360 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:16:17.175360 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:17.175360 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:17.175360 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:17.175360 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:17.175360 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:17.175360 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:17.175360 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:17.175360 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:17.175360 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:17.175360 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:17.175360 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:17.175360 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:17.175360 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:17.175360 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:17.175360 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:17.175360 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:17.176514 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:16:17.176514 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:16:17.176514 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:17.176514 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:17.179019 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 481/939] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:16:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:17.184472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee51d5-ecef-422e-926a-33823e04b423 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:17.295350 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 459/940] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Jul 7 15:16:17 2022] POST /identity/v3/endpoints => generated 347 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:16:17.302596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee51d5-ecef-422e-926a-33823e04b423 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:17.315597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:17.354891 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e43aff-6a73-4123-8769-3de2ec3e89b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:17.355589 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e43aff-6a73-4123-8769-3de2ec3e89b8 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:17.355882 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e43aff-6a73-4123-8769-3de2ec3e89b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:17.356157 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e43aff-6a73-4123-8769-3de2ec3e89b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:17.357468 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e43aff-6a73-4123-8769-3de2ec3e89b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:17.357782 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e43aff-6a73-4123-8769-3de2ec3e89b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:17.359227 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e43aff-6a73-4123-8769-3de2ec3e89b8 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:17.374225 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 482/941] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:16:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:17.380187 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5ce40-9856-442e-a285-694ad5c5d80a None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:17.490797 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:17.499518 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5ce40-9856-442e-a285-694ad5c5d80a None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:17.536871 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80cb26e-51b5-4237-b64b-a2679e9d2f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:17.537673 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80cb26e-51b5-4237-b64b-a2679e9d2f80 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:17.537956 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80cb26e-51b5-4237-b64b-a2679e9d2f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:17.538233 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80cb26e-51b5-4237-b64b-a2679e9d2f80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:17.546230 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80cb26e-51b5-4237-b64b-a2679e9d2f80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:17.547629 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80cb26e-51b5-4237-b64b-a2679e9d2f80 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:17.563018 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-c80cb26e-51b5-4237-b64b-a2679e9d2f80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:16:17.563018 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:17.563018 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:17.563018 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:17.563018 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:17.563018 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:17.563018 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:17.563018 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:17.563018 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:17.563018 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:17.563018 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:17.563018 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:17.563018 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 15:16:17.563018 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 15:16:17.563018 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 15:16:17.564192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 15:16:17.564192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:17.564192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:17.564192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:17.564192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:17.564192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:17.564192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:17.564192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:17.564192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:17.564192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:17.564192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:17.564192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:17.564192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:17.564192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:17.564192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:17.564192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:17.564192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:17.565249 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 15:16:17.565249 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 15:16:17.565249 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 15:16:17.565249 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 15:16:17.565249 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 15:16:17.565249 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 15:16:17.565249 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:16:17.565249 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:17.567942 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 460/942] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:16:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:17.573291 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e43aff-6a73-4123-8769-3de2ec3e89b8 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:17.652297 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03752a1-ef87-42b3-9a24-31d4140db589 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:17.652639 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03752a1-ef87-42b3-9a24-31d4140db589 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:17.652865 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03752a1-ef87-42b3-9a24-31d4140db589 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:17.655030 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 461/943] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:16:17.660001 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b9ac66-7a31-4a1a-80d6-03b61cf34ff5 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:17.660289 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b9ac66-7a31-4a1a-80d6-03b61cf34ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:17.660569 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b9ac66-7a31-4a1a-80d6-03b61cf34ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:17.680598 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-22b9ac66-7a31-4a1a-80d6-03b61cf34ff5 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:17.681795 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-22b9ac66-7a31-4a1a-80d6-03b61cf34ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:17.685737 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e43aff-6a73-4123-8769-3de2ec3e89b8 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:17.723053 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32205498-48b3-4f1f-bfa7-b1ae02b86488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:17.723833 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32205498-48b3-4f1f-bfa7-b1ae02b86488 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:17.724018 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32205498-48b3-4f1f-bfa7-b1ae02b86488 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:17.724294 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32205498-48b3-4f1f-bfa7-b1ae02b86488 None admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:17.725210 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32205498-48b3-4f1f-bfa7-b1ae02b86488 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:17.725521 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32205498-48b3-4f1f-bfa7-b1ae02b86488 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:17.727068 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32205498-48b3-4f1f-bfa7-b1ae02b86488 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:17.740916 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 483/944] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Jul 7 15:16:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:17.747521 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32205498-48b3-4f1f-bfa7-b1ae02b86488 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:17.759648 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 462/945] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 07 15:16:17.772135 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:17.787538 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b9ac66-7a31-4a1a-80d6-03b61cf34ff5 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:17.870074 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32205498-48b3-4f1f-bfa7-b1ae02b86488 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:17.903238 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:17.909653 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec4b7a-05ec-4b9a-8982-a954d3a3e422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:17.910753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec4b7a-05ec-4b9a-8982-a954d3a3e422 None admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:17.911308 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec4b7a-05ec-4b9a-8982-a954d3a3e422 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:17.911765 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec4b7a-05ec-4b9a-8982-a954d3a3e422 None admin] PATH_INFO: `/v3/projects/c5260c7fd3614e9e93b167c86a94260e/users/cd3c04f39d774998b66e6ce920925691/roles/d5899b9e6b9940aa9837652e51a23908` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:17.917715 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ec4b7a-05ec-4b9a-8982-a954d3a3e422 None admin] RBAC: Authorizing `identity:create_grant(project_id=c5260c7fd3614e9e93b167c86a94260e, user_id=cd3c04f39d774998b66e6ce920925691, role_id=d5899b9e6b9940aa9837652e51a23908)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:17.917715 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ec4b7a-05ec-4b9a-8982-a954d3a3e422 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:17.917914 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b9ac66-7a31-4a1a-80d6-03b61cf34ff5 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:17.959418 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21164125-b739-4233-a725-b56080256c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:17.960175 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21164125-b739-4233-a725-b56080256c6b None admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:17.961952 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21164125-b739-4233-a725-b56080256c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:17.961952 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21164125-b739-4233-a725-b56080256c6b None admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:17.961952 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21164125-b739-4233-a725-b56080256c6b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:17.962822 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21164125-b739-4233-a725-b56080256c6b None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:17.964354 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: WARNING keystone.server.flask.common [None req-21164125-b739-4233-a725-b56080256c6b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 07 15:16:18.015653 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 484/946] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Jul 7 15:16:17 2022] PUT /identity/v3/projects/c5260c7fd3614e9e93b167c86a94260e/users/cd3c04f39d774998b66e6ce920925691/roles/d5899b9e6b9940aa9837652e51a23908 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:16:18.025717 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.sql.core [None req-21164125-b739-4233-a725-b56080256c6b None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 07 15:16:18.025717 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 07 15:16:18.025717 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [parameters: {'id': '255cceacd5fb4477a72e78d51952562e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 07 15:16:18.025717 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80300) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 07 15:16:18.034803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-21164125-b739-4233-a725-b56080256c6b None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 255cceacd5fb4477a72e78d51952562e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 255cceacd5fb4477a72e78d51952562e. Jul 07 15:16:18.034803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:18.034803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 07 15:16:18.034803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 07 15:16:18.034803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 07 15:16:18.034803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 07 15:16:18.034803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 07 15:16:18.034803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application result = self._query(query) Jul 07 15:16:18.034803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 07 15:16:18.034803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application conn.query(q) Jul 07 15:16:18.034803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 07 15:16:18.034803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 07 15:16:18.034803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 07 15:16:18.039210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application result.read() Jul 07 15:16:18.039210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 07 15:16:18.039210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 07 15:16:18.039210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 07 15:16:18.039210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 07 15:16:18.039210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 07 15:16:18.039210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 07 15:16:18.039210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 07 15:16:18.039210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 07 15:16:18.039210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 07 15:16:18.039210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:18.039210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 07 15:16:18.039210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:18.039210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:18.039210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 07 15:16:18.039210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 07 15:16:18.039210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 07 15:16:18.039210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 07 15:16:18.040311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 07 15:16:18.040311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application next(self.gen) Jul 07 15:16:18.040311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 07 15:16:18.040311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application yield resource Jul 07 15:16:18.040311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 07 15:16:18.040311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application next(self.gen) Jul 07 15:16:18.040311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 07 15:16:18.040311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self.session.rollback() Jul 07 15:16:18.040311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 07 15:16:18.040311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self.force_reraise() Jul 07 15:16:18.040311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 07 15:16:18.040311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise self.value Jul 07 15:16:18.040311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 07 15:16:18.040311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 07 15:16:18.040311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 07 15:16:18.040311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application session.commit() Jul 07 15:16:18.040311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 07 15:16:18.041229 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 07 15:16:18.041229 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 07 15:16:18.041229 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self._prepare_impl() Jul 07 15:16:18.041229 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 07 15:16:18.041229 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self.session.flush() Jul 07 15:16:18.041229 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 07 15:16:18.041229 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self._flush(objects) Jul 07 15:16:18.041229 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 07 15:16:18.041229 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 07 15:16:18.041229 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 07 15:16:18.041229 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application compat.raise_( Jul 07 15:16:18.041229 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 07 15:16:18.041229 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception Jul 07 15:16:18.041229 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 07 15:16:18.041229 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application flush_context.execute() Jul 07 15:16:18.041229 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 07 15:16:18.041229 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rec.execute(self) Jul 07 15:16:18.042135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 07 15:16:18.042135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 07 15:16:18.042135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 07 15:16:18.042135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 07 15:16:18.042135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 07 15:16:18.042135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 07 15:16:18.042135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 07 15:16:18.042135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 07 15:16:18.042135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 07 15:16:18.042135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 07 15:16:18.042135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 07 15:16:18.042135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 07 15:16:18.042135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 07 15:16:18.042135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 07 15:16:18.042135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 07 15:16:18.042940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 07 15:16:18.042940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 07 15:16:18.042940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception Jul 07 15:16:18.042940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 07 15:16:18.042940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 07 15:16:18.042940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 07 15:16:18.042940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 07 15:16:18.042940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 07 15:16:18.042940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application result = self._query(query) Jul 07 15:16:18.042940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 07 15:16:18.042940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application conn.query(q) Jul 07 15:16:18.042940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 07 15:16:18.042940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 07 15:16:18.042940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 07 15:16:18.042940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application result.read() Jul 07 15:16:18.042940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 07 15:16:18.042940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 07 15:16:18.043932 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 07 15:16:18.043932 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 07 15:16:18.043932 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 07 15:16:18.043932 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 07 15:16:18.043932 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 07 15:16:18.043932 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 07 15:16:18.043932 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 07 15:16:18.043932 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 07 15:16:18.043932 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [parameters: {'id': '255cceacd5fb4477a72e78d51952562e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 07 15:16:18.043932 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 07 15:16:18.043932 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:18.043932 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 15:16:18.043932 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:18.043932 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:18.043932 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 07 15:16:18.048295 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 07 15:16:18.048295 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 07 15:16:18.048295 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 07 15:16:18.048295 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 07 15:16:18.048295 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:18.048295 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 15:16:18.048295 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:18.048295 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:18.048295 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:18.048295 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:18.048295 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:18.048295 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:18.048295 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:18.048295 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:18.048295 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:18.048295 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:18.048295 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:18.050353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:18.050353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 07 15:16:18.050353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 07 15:16:18.050353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:18.050353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:18.050353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 07 15:16:18.050353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 07 15:16:18.050353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 255cceacd5fb4477a72e78d51952562e. Jul 07 15:16:18.050353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:18.050353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 463/947] 10.222.0.25 () {60 vars in 1140 bytes} [Thu Jul 7 15:16:17 2022] POST /identity/v3/projects => generated 262 bytes in 250 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:18.051481 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec4b7a-05ec-4b9a-8982-a954d3a3e422 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:18.161497 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:18.164326 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec4b7a-05ec-4b9a-8982-a954d3a3e422 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:18.202299 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2539f1a6-8051-4f59-9a7e-b66b42acf934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:18.203170 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539f1a6-8051-4f59-9a7e-b66b42acf934 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:18.203424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539f1a6-8051-4f59-9a7e-b66b42acf934 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:18.203717 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539f1a6-8051-4f59-9a7e-b66b42acf934 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:18.210609 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2539f1a6-8051-4f59-9a7e-b66b42acf934 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:18.212080 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2539f1a6-8051-4f59-9a7e-b66b42acf934 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:18.226781 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-2539f1a6-8051-4f59-9a7e-b66b42acf934 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:16:18.226781 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:18.226781 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:18.226781 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:18.226781 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:18.226781 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:18.226781 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:18.226781 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:18.226781 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:18.226781 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:18.226781 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:18.226781 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:18.226781 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 15:16:18.226781 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 15:16:18.226781 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 15:16:18.227720 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 15:16:18.227720 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:18.227720 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:18.227720 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:18.227720 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:18.227720 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:18.227720 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:18.227720 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:18.227720 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:18.227720 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:18.227720 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:18.227720 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:18.227720 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:18.227720 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:18.227720 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:18.227720 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:18.227720 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:18.228638 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 15:16:18.228638 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 15:16:18.228638 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 15:16:18.228638 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 15:16:18.228638 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 15:16:18.228638 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 15:16:18.228638 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:16:18.228638 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:18.228638 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 485/948] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:16:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:18.236875 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21164125-b739-4233-a725-b56080256c6b None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:18.343722 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21164125-b739-4233-a725-b56080256c6b None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:18.380082 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2eb541-facd-44c0-a8c3-eb60a1c83043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:18.381115 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2eb541-facd-44c0-a8c3-eb60a1c83043 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:18.381623 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2eb541-facd-44c0-a8c3-eb60a1c83043 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:18.382107 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2eb541-facd-44c0-a8c3-eb60a1c83043 None admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:18.383077 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2eb541-facd-44c0-a8c3-eb60a1c83043 None admin] RBAC: Adding query filter params (name=service) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:18.383626 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2eb541-facd-44c0-a8c3-eb60a1c83043 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:18.385599 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2eb541-facd-44c0-a8c3-eb60a1c83043 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:18.402153 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:18.409872 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 464/949] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Jul 7 15:16:18 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:18.884628 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5baab4-da12-46cb-bc50-38a9abd206ed None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:18.890622 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5baab4-da12-46cb-bc50-38a9abd206ed None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:18.890622 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5baab4-da12-46cb-bc50-38a9abd206ed None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:18.890622 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 486/950] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:16:18.892119 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87b8dc-e676-4a3b-a52c-d4788c246aa7 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:18.892526 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87b8dc-e676-4a3b-a52c-d4788c246aa7 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:18.892706 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87b8dc-e676-4a3b-a52c-d4788c246aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:18.909801 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-5e87b8dc-e676-4a3b-a52c-d4788c246aa7 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:18.911320 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-5e87b8dc-e676-4a3b-a52c-d4788c246aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:18.987463 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 465/951] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:16:19.011306 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5baab4-da12-46cb-bc50-38a9abd206ed None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:19.124767 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5baab4-da12-46cb-bc50-38a9abd206ed None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:19.160808 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97797f6c-7f16-4389-a109-29e7aa6d846c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:19.161538 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97797f6c-7f16-4389-a109-29e7aa6d846c None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:19.161825 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97797f6c-7f16-4389-a109-29e7aa6d846c None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:19.162106 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97797f6c-7f16-4389-a109-29e7aa6d846c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:19.162761 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97797f6c-7f16-4389-a109-29e7aa6d846c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:19.164168 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97797f6c-7f16-4389-a109-29e7aa6d846c None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:19.176875 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-97797f6c-7f16-4389-a109-29e7aa6d846c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 07 15:16:19.176875 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:19.176875 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 15:16:19.176875 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 15:16:19.176875 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 15:16:19.176875 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 15:16:19.176875 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 15:16:19.176875 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:16:19.176875 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 07 15:16:19.176875 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:19.176875 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 15:16:19.176875 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:19.176875 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:19.176875 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:19.176875 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:19.176875 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:19.176875 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:19.178030 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:19.178030 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:19.178030 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:19.178030 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:19.178030 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:19.178030 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:19.178030 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 15:16:19.178030 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 15:16:19.178030 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 15:16:19.178030 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 15:16:19.178030 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:19.178030 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:19.178030 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:19.178030 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:19.178030 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:19.178030 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:19.178944 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:19.178944 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:19.178944 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:19.178944 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:19.178944 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:19.178944 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:19.178944 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:19.178944 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:19.178944 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:19.178944 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:19.178944 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 15:16:19.178944 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:16:19.178944 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 07 15:16:19.178944 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:19.178944 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 487/952] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Jul 7 15:16:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:19.186611 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e87b8dc-e676-4a3b-a52c-d4788c246aa7 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:19.293729 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e87b8dc-e676-4a3b-a52c-d4788c246aa7 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:19.323802 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:19.329628 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d01dd-671a-4d5a-8a75-a4dbfc6b993f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:19.330289 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d01dd-671a-4d5a-8a75-a4dbfc6b993f None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:19.330576 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d01dd-671a-4d5a-8a75-a4dbfc6b993f None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:19.330854 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d01dd-671a-4d5a-8a75-a4dbfc6b993f None admin] PATH_INFO: `/v3/services` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:19.331576 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d01dd-671a-4d5a-8a75-a4dbfc6b993f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:19.331894 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d01dd-671a-4d5a-8a75-a4dbfc6b993f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:19.333249 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d01dd-671a-4d5a-8a75-a4dbfc6b993f None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:19.338315 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 466/953] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Jul 7 15:16:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:19.344635 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97797f6c-7f16-4389-a109-29e7aa6d846c None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:19.348043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:19.446398 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12239131-9737-49b0-8d09-5923e4660009 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:19.446585 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12239131-9737-49b0-8d09-5923e4660009 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:19.446903 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12239131-9737-49b0-8d09-5923e4660009 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:19.448560 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 467/954] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:16:19.455860 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed1729-9fdb-40cc-be8a-7f4bf3c9399f None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:19.456252 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed1729-9fdb-40cc-be8a-7f4bf3c9399f None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:19.456632 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed1729-9fdb-40cc-be8a-7f4bf3c9399f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:19.457019 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97797f6c-7f16-4389-a109-29e7aa6d846c None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:19.477953 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-e3ed1729-9fdb-40cc-be8a-7f4bf3c9399f None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:19.479961 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-e3ed1729-9fdb-40cc-be8a-7f4bf3c9399f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:19.493097 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb8c88-7173-406e-b1b1-220d95b99b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:19.493972 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb8c88-7173-406e-b1b1-220d95b99b72 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:19.494255 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb8c88-7173-406e-b1b1-220d95b99b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:19.494528 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb8c88-7173-406e-b1b1-220d95b99b72 None admin] PATH_INFO: `/v3/services` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:19.495285 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bb8c88-7173-406e-b1b1-220d95b99b72 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:19.495624 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bb8c88-7173-406e-b1b1-220d95b99b72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:19.496992 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bb8c88-7173-406e-b1b1-220d95b99b72 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:19.502352 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 488/955] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Jul 7 15:16:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:19.508774 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb8c88-7173-406e-b1b1-220d95b99b72 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:19.559643 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 468/956] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:16:19.580968 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:19.583068 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ed1729-9fdb-40cc-be8a-7f4bf3c9399f None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:19.646192 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb8c88-7173-406e-b1b1-220d95b99b72 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:19.702373 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9153515a-86f0-4fc6-8a0c-05a41f0faeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:19.707588 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153515a-86f0-4fc6-8a0c-05a41f0faeea None admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:19.707588 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153515a-86f0-4fc6-8a0c-05a41f0faeea None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:19.707588 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153515a-86f0-4fc6-8a0c-05a41f0faeea None admin] PATH_INFO: `/v3/endpoints` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:19.707588 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9153515a-86f0-4fc6-8a0c-05a41f0faeea None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:19.707588 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9153515a-86f0-4fc6-8a0c-05a41f0faeea None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:19.779607 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ed1729-9fdb-40cc-be8a-7f4bf3c9399f None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:19.802416 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:19.813125 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 489/957] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Jul 7 15:16:19 2022] POST /identity/v3/endpoints => generated 349 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:16:19.846759 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdae894-fb04-47f2-9bb4-95ec625735cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:19.849188 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdae894-fb04-47f2-9bb4-95ec625735cf None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:19.849723 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdae894-fb04-47f2-9bb4-95ec625735cf None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:19.850203 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdae894-fb04-47f2-9bb4-95ec625735cf None admin] PATH_INFO: `/v3/services/compute` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:19.851116 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdae894-fb04-47f2-9bb4-95ec625735cf None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:19.853938 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdae894-fb04-47f2-9bb4-95ec625735cf None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:19.876915 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-6fdae894-fb04-47f2-9bb4-95ec625735cf None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 07 15:16:19.876915 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:19.876915 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 15:16:19.876915 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 15:16:19.876915 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 15:16:19.876915 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 15:16:19.876915 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 15:16:19.876915 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:16:19.876915 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 07 15:16:19.876915 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:19.876915 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 15:16:19.876915 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:19.876915 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:19.876915 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:19.876915 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:19.876915 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:19.876915 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:19.884769 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:19.884769 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:19.884769 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:19.884769 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:19.884769 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:19.884769 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:19.884769 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 15:16:19.884769 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 15:16:19.884769 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 15:16:19.884769 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 15:16:19.884769 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:19.884769 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:19.884769 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:19.884769 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:19.884769 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:19.884769 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:19.885684 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:19.885684 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:19.885684 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:19.885684 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:19.885684 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:19.885684 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:19.885684 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:19.885684 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:19.885684 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:19.885684 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:19.885684 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 15:16:19.885684 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:16:19.885684 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 07 15:16:19.885684 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:19.885684 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 469/958] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:16:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:16:19.895320 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9153515a-86f0-4fc6-8a0c-05a41f0faeea None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:20.066671 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9153515a-86f0-4fc6-8a0c-05a41f0faeea None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:20.093749 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:20.115481 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359867db-76d3-4ec8-abc2-1843b7f09db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:20.116205 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359867db-76d3-4ec8-abc2-1843b7f09db4 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:20.119030 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359867db-76d3-4ec8-abc2-1843b7f09db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:20.119030 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359867db-76d3-4ec8-abc2-1843b7f09db4 None admin] PATH_INFO: `/v3/services` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:20.119030 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359867db-76d3-4ec8-abc2-1843b7f09db4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:20.119030 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359867db-76d3-4ec8-abc2-1843b7f09db4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:20.119421 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359867db-76d3-4ec8-abc2-1843b7f09db4 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:20.127788 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 490/959] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Jul 7 15:16:19 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:20.132755 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdae894-fb04-47f2-9bb4-95ec625735cf None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:20.240133 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdae894-fb04-47f2-9bb4-95ec625735cf None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:20.255286 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c98984-7a14-4787-aee8-2ae6599f3753 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:20.255600 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c98984-7a14-4787-aee8-2ae6599f3753 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:20.255862 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c98984-7a14-4787-aee8-2ae6599f3753 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:20.257718 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 491/960] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:16:20.266283 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103f5b2-9b3e-43e4-bc9a-024a95cce3d0 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:20.266844 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103f5b2-9b3e-43e4-bc9a-024a95cce3d0 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:20.267350 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103f5b2-9b3e-43e4-bc9a-024a95cce3d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:20.282898 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a049e4f7-7756-4c46-b433-a80f57894cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:20.283601 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049e4f7-7756-4c46-b433-a80f57894cfc None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:20.283872 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049e4f7-7756-4c46-b433-a80f57894cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:20.284142 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049e4f7-7756-4c46-b433-a80f57894cfc None admin] PATH_INFO: `/v3/services` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:20.284853 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a049e4f7-7756-4c46-b433-a80f57894cfc None admin] RBAC: Adding query filter params (type=compute) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:20.285181 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a049e4f7-7756-4c46-b433-a80f57894cfc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:20.286297 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-9103f5b2-9b3e-43e4-bc9a-024a95cce3d0 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:20.286491 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a049e4f7-7756-4c46-b433-a80f57894cfc None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:20.287915 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-9103f5b2-9b3e-43e4-bc9a-024a95cce3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:20.292326 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 470/961] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Jul 7 15:16:20 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:20.372363 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 492/962] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 07 15:16:20.397045 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a049e4f7-7756-4c46-b433-a80f57894cfc None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:20.459923 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:20.513194 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a049e4f7-7756-4c46-b433-a80f57894cfc None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:20.547196 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b998017-85d3-463b-9ce3-0469d9eb1fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:20.548033 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b998017-85d3-463b-9ce3-0469d9eb1fad None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:20.548314 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b998017-85d3-463b-9ce3-0469d9eb1fad None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:20.548587 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b998017-85d3-463b-9ce3-0469d9eb1fad None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:20.558931 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b998017-85d3-463b-9ce3-0469d9eb1fad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:16:20.567805 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-1b998017-85d3-463b-9ce3-0469d9eb1fad None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 07 15:16:20.567805 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:20.567805 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:20.567805 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:20.567805 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:20.567805 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:20.567805 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:20.567805 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:20.567805 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:20.567805 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:20.567805 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:20.567805 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:20.567805 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:16:20.567805 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:16:20.567805 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:16:20.567805 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:16:20.568787 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:16:20.568787 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:16:20.568787 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:16:20.568787 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:16:20.568787 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:20.568787 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:20.568787 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:20.568787 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:20.568787 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:20.568787 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:20.568787 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:20.568787 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:20.568787 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:20.568787 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:20.568787 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:20.568787 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:20.569669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:20.569669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:20.569669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:20.569669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:20.569669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:16:20.569669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:16:20.569669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:16:20.569669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:16:20.569669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:16:20.569669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:16:20.569669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 07 15:16:20.569669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:20.569669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 471/963] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Jul 7 15:16:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:20.576649 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103f5b2-9b3e-43e4-bc9a-024a95cce3d0 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:20.591945 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66f804-380f-43e7-9052-8de9a005fe5c None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:20.592248 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66f804-380f-43e7-9052-8de9a005fe5c None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:20.592586 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66f804-380f-43e7-9052-8de9a005fe5c None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:20.594115 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 472/964] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:16:20.599188 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89123d1-900d-41c2-b483-e44830c9df28 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:20.599476 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89123d1-900d-41c2-b483-e44830c9df28 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:20.599789 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89123d1-900d-41c2-b483-e44830c9df28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:20.617671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-b89123d1-900d-41c2-b483-e44830c9df28 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:20.618856 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-b89123d1-900d-41c2-b483-e44830c9df28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:20.688481 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 473/965] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 07 15:16:20.691231 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103f5b2-9b3e-43e4-bc9a-024a95cce3d0 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:20.712111 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:20.712559 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89123d1-900d-41c2-b483-e44830c9df28 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:20.733791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f036921-5503-4095-bfb5-6a948930bcde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:20.734415 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f036921-5503-4095-bfb5-6a948930bcde None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:20.734704 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f036921-5503-4095-bfb5-6a948930bcde None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:20.734981 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f036921-5503-4095-bfb5-6a948930bcde None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:20.735761 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f036921-5503-4095-bfb5-6a948930bcde None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:20.736070 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f036921-5503-4095-bfb5-6a948930bcde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:20.737511 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f036921-5503-4095-bfb5-6a948930bcde None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:20.747134 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 493/966] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Jul 7 15:16:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:20.753735 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f036921-5503-4095-bfb5-6a948930bcde None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:20.758387 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:20.830792 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89123d1-900d-41c2-b483-e44830c9df28 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:20.861526 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f036921-5503-4095-bfb5-6a948930bcde None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:20.869918 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e6ea47-09a4-4748-91e5-223538daeb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:20.870597 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e6ea47-09a4-4748-91e5-223538daeb09 None admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:20.870976 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e6ea47-09a4-4748-91e5-223538daeb09 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:20.871264 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e6ea47-09a4-4748-91e5-223538daeb09 None admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:20.872002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e6ea47-09a4-4748-91e5-223538daeb09 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:20.873574 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e6ea47-09a4-4748-91e5-223538daeb09 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:20.897502 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec370bc2-fcc5-4fbf-892e-65e1b94b027c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:20.898292 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec370bc2-fcc5-4fbf-892e-65e1b94b027c None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:20.898581 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec370bc2-fcc5-4fbf-892e-65e1b94b027c None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:20.898859 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec370bc2-fcc5-4fbf-892e-65e1b94b027c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:20.910803 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec370bc2-fcc5-4fbf-892e-65e1b94b027c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:20.912275 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec370bc2-fcc5-4fbf-892e-65e1b94b027c None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:20.930748 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-ec370bc2-fcc5-4fbf-892e-65e1b94b027c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:20.930748 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:20.930748 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:20.930748 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:20.930748 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:20.930748 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:20.930748 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:20.930748 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:20.930748 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:20.930748 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:20.930748 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:20.930748 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:20.930748 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:16:20.930748 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:16:20.930748 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:16:20.932000 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:16:20.932000 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:20.932000 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:20.932000 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:20.932000 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:20.932000 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:20.932000 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:20.932000 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:20.932000 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:20.932000 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:20.932000 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:20.932000 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:20.932000 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:20.932000 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:20.932000 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:20.932000 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:20.932000 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:20.932914 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:16:20.932914 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:16:20.932914 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:20.932914 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:20.933524 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 494/967] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:16:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:16:20.937542 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec370bc2-fcc5-4fbf-892e-65e1b94b027c None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:20.939727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.sql.core [None req-76e6ea47-09a4-4748-91e5-223538daeb09 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 07 15:16:20.939727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 07 15:16:20.939727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [parameters: {'user_id': '06c40acf175a474cb9d975c896e33d78', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 07 15:16:20.939727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80300) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 07 15:16:20.947533 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-76e6ea47-09a4-4748-91e5-223538daeb09 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 07 15:16:20.947533 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:20.947533 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 07 15:16:20.947533 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 07 15:16:20.947533 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 07 15:16:20.947533 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 07 15:16:20.947533 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 07 15:16:20.947533 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application result = self._query(query) Jul 07 15:16:20.947533 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 07 15:16:20.947533 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application conn.query(q) Jul 07 15:16:20.947533 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 07 15:16:20.947533 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 07 15:16:20.947533 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 07 15:16:20.947533 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application result.read() Jul 07 15:16:20.947533 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 07 15:16:20.948474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 07 15:16:20.948474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 07 15:16:20.948474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 07 15:16:20.948474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 07 15:16:20.948474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 07 15:16:20.948474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 07 15:16:20.948474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 07 15:16:20.948474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 07 15:16:20.948474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:20.948474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 07 15:16:20.948474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:20.948474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:20.948474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 07 15:16:20.948474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 07 15:16:20.948474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 07 15:16:20.948474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 07 15:16:20.948474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 07 15:16:20.948474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application next(self.gen) Jul 07 15:16:20.949513 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 07 15:16:20.949513 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application yield resource Jul 07 15:16:20.949513 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 07 15:16:20.949513 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application next(self.gen) Jul 07 15:16:20.949513 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 07 15:16:20.949513 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self.session.rollback() Jul 07 15:16:20.949513 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 07 15:16:20.949513 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self.force_reraise() Jul 07 15:16:20.949513 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 07 15:16:20.949513 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise self.value Jul 07 15:16:20.949513 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 07 15:16:20.949513 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 07 15:16:20.949513 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 07 15:16:20.949513 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application session.commit() Jul 07 15:16:20.949513 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 07 15:16:20.949513 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 07 15:16:20.949513 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 07 15:16:20.951506 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self._prepare_impl() Jul 07 15:16:20.951506 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 07 15:16:20.951506 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self.session.flush() Jul 07 15:16:20.951506 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 07 15:16:20.951506 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self._flush(objects) Jul 07 15:16:20.951506 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 07 15:16:20.951506 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 07 15:16:20.951506 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 07 15:16:20.951506 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application compat.raise_( Jul 07 15:16:20.951506 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 07 15:16:20.951506 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception Jul 07 15:16:20.951506 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 07 15:16:20.951506 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application flush_context.execute() Jul 07 15:16:20.951506 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 07 15:16:20.951506 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rec.execute(self) Jul 07 15:16:20.951506 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 07 15:16:20.951506 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 07 15:16:20.954061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 07 15:16:20.954061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 07 15:16:20.954061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 07 15:16:20.954061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 07 15:16:20.954061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 07 15:16:20.954061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 07 15:16:20.954061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 07 15:16:20.954061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 07 15:16:20.954061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 07 15:16:20.954061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 07 15:16:20.954061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 07 15:16:20.954061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 07 15:16:20.954061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 07 15:16:20.954061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 07 15:16:20.954061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 07 15:16:20.954061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception Jul 07 15:16:20.956212 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 07 15:16:20.956212 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 07 15:16:20.956212 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 07 15:16:20.956212 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 07 15:16:20.956212 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 07 15:16:20.956212 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application result = self._query(query) Jul 07 15:16:20.956212 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 07 15:16:20.956212 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application conn.query(q) Jul 07 15:16:20.956212 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 07 15:16:20.956212 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 07 15:16:20.956212 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 07 15:16:20.956212 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application result.read() Jul 07 15:16:20.956212 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 07 15:16:20.956212 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 07 15:16:20.956212 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 07 15:16:20.956212 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 07 15:16:20.958343 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 07 15:16:20.958343 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 07 15:16:20.958343 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 07 15:16:20.958343 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 07 15:16:20.958343 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 07 15:16:20.958343 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 07 15:16:20.958343 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [parameters: {'user_id': '06c40acf175a474cb9d975c896e33d78', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 07 15:16:20.958343 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 07 15:16:20.958343 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:20.958343 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 15:16:20.958343 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:20.958343 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:20.958343 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:20.958343 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:20.958343 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:20.960317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:20.960317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:20.960317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:20.960317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:20.960317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:20.960317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:20.960317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:20.960317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 07 15:16:20.960317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 07 15:16:20.960317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:20.960317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:20.960317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 15:16:20.960317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:16:20.960317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 15:16:20.960317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:16:20.960317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 07 15:16:20.960317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 07 15:16:20.964854 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 07 15:16:20.964854 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 07 15:16:20.964854 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 07 15:16:20.964854 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 07 15:16:20.964854 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 07 15:16:20.964854 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:20.964854 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 474/968] 10.222.0.25 () {60 vars in 1131 bytes} [Thu Jul 7 15:16:20 2022] POST /identity/v3/users => generated 160 bytes in 237 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:20.964854 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e6ea47-09a4-4748-91e5-223538daeb09 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:20.995598 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:21.072137 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec370bc2-fcc5-4fbf-892e-65e1b94b027c None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:21.072355 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:21.076659 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e6ea47-09a4-4748-91e5-223538daeb09 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:21.113982 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724c578-a9b8-4e94-9233-ddd660ada1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:21.115051 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724c578-a9b8-4e94-9233-ddd660ada1bd None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:21.115490 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724c578-a9b8-4e94-9233-ddd660ada1bd None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:21.115640 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67919399-601f-439f-9607-f8e25a60bb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:21.116122 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724c578-a9b8-4e94-9233-ddd660ada1bd None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:21.116657 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67919399-601f-439f-9607-f8e25a60bb5e None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:21.117037 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67919399-601f-439f-9607-f8e25a60bb5e None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:21.117358 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a724c578-a9b8-4e94-9233-ddd660ada1bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:21.117483 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67919399-601f-439f-9607-f8e25a60bb5e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:21.117925 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a724c578-a9b8-4e94-9233-ddd660ada1bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:21.119699 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a724c578-a9b8-4e94-9233-ddd660ada1bd None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:21.125745 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67919399-601f-439f-9607-f8e25a60bb5e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:21.127678 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67919399-601f-439f-9607-f8e25a60bb5e None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:21.139093 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 495/969] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:21.140357 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724c578-a9b8-4e94-9233-ddd660ada1bd None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:21.145140 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-67919399-601f-439f-9607-f8e25a60bb5e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 07 15:16:21.145140 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:21.145140 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:21.145140 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:21.145140 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:21.145140 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:21.145140 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:21.145140 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:21.145140 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:21.145140 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:21.145140 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:21.145140 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:21.145140 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 15:16:21.145140 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 15:16:21.145140 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 15:16:21.145140 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 15:16:21.146421 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:21.146421 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:21.146421 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 15:16:21.146421 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:16:21.146421 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 15:16:21.146421 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:16:21.146421 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:21.146421 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:21.146421 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:21.146421 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:21.146421 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:21.146421 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:21.146421 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:21.146421 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:21.146421 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:21.146421 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:21.146421 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:21.146421 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:21.147393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:21.147393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:21.147393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 15:16:21.147393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 15:16:21.147393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 15:16:21.147393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 15:16:21.147393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 15:16:21.147393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 15:16:21.147393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 07 15:16:21.147393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:21.151150 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 475/970] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:16:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:21.152242 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67919399-601f-439f-9607-f8e25a60bb5e None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:21.241741 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:21.252430 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724c578-a9b8-4e94-9233-ddd660ada1bd None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:21.257358 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67919399-601f-439f-9607-f8e25a60bb5e None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:21.289477 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b4074e-138e-4299-96d1-f7f486303099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:21.290913 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4074e-138e-4299-96d1-f7f486303099 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:21.291555 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4074e-138e-4299-96d1-f7f486303099 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:21.292072 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4074e-138e-4299-96d1-f7f486303099 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:21.293033 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a7c46-a890-4c84-bf6f-db0f282977d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:21.293883 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a7c46-a890-4c84-bf6f-db0f282977d5 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:21.294314 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a7c46-a890-4c84-bf6f-db0f282977d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:21.294699 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a7c46-a890-4c84-bf6f-db0f282977d5 None admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:21.295627 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a7c46-a890-4c84-bf6f-db0f282977d5 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:21.296088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a7c46-a890-4c84-bf6f-db0f282977d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:21.297601 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a7c46-a890-4c84-bf6f-db0f282977d5 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:21.302907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b4074e-138e-4299-96d1-f7f486303099 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:21.304909 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b4074e-138e-4299-96d1-f7f486303099 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:21.320768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-c2b4074e-138e-4299-96d1-f7f486303099 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 07 15:16:21.320768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:21.320768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:21.320768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:21.320768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:21.320768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:21.320768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:21.320768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:21.320768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:21.320768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:21.320768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:21.320768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:21.320768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 15:16:21.320768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 15:16:21.320768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 15:16:21.320768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 15:16:21.321973 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:21.321973 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:21.321973 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 15:16:21.321973 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:16:21.321973 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 15:16:21.321973 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:16:21.321973 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:21.321973 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:21.321973 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:21.321973 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:21.321973 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:21.321973 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:21.321973 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:21.321973 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:21.321973 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:21.321973 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:21.321973 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:21.321973 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:21.323277 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:21.323277 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:21.323277 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 15:16:21.323277 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 15:16:21.323277 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 15:16:21.323277 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 15:16:21.323277 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 15:16:21.323277 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 15:16:21.323277 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 07 15:16:21.323277 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:21.330796 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 496/971] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Jul 7 15:16:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:21.331810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b4074e-138e-4299-96d1-f7f486303099 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:21.347967 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 476/972] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Jul 7 15:16:21 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:21.450300 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b4074e-138e-4299-96d1-f7f486303099 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:21.493751 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f1a0c8-483d-4dd3-8b07-2bf979c7d041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:21.494674 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f1a0c8-483d-4dd3-8b07-2bf979c7d041 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:21.495116 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f1a0c8-483d-4dd3-8b07-2bf979c7d041 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:21.495581 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f1a0c8-483d-4dd3-8b07-2bf979c7d041 None admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:21.496580 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f1a0c8-483d-4dd3-8b07-2bf979c7d041 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:21.497041 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f1a0c8-483d-4dd3-8b07-2bf979c7d041 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:21.498667 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f1a0c8-483d-4dd3-8b07-2bf979c7d041 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:21.518274 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 497/973] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Jul 7 15:16:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:21.524495 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a7c46-a890-4c84-bf6f-db0f282977d5 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:21.551115 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:21.642776 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a7c46-a890-4c84-bf6f-db0f282977d5 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:21.678279 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead36bc-d116-427a-8fce-64061c10f19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:21.678918 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead36bc-d116-427a-8fce-64061c10f19b None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:21.679194 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead36bc-d116-427a-8fce-64061c10f19b None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:21.679466 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead36bc-d116-427a-8fce-64061c10f19b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:21.747668 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dead36bc-d116-427a-8fce-64061c10f19b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:21.749029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dead36bc-d116-427a-8fce-64061c10f19b None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:21.767483 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:21.851614 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 477/974] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:16:21 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 07 15:16:21.852878 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f1a0c8-483d-4dd3-8b07-2bf979c7d041 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:21.863592 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:21.960649 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f1a0c8-483d-4dd3-8b07-2bf979c7d041 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:21.997345 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f84847f-cf7d-41cf-b5b6-b09fe51a6d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:21.998216 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84847f-cf7d-41cf-b5b6-b09fe51a6d84 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:21.998438 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84847f-cf7d-41cf-b5b6-b09fe51a6d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:21.998703 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84847f-cf7d-41cf-b5b6-b09fe51a6d84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:22.009045 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f84847f-cf7d-41cf-b5b6-b09fe51a6d84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:22.010488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f84847f-cf7d-41cf-b5b6-b09fe51a6d84 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:22.029531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-7f84847f-cf7d-41cf-b5b6-b09fe51a6d84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:22.029531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:22.029531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:22.029531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:22.029531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:22.029531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:22.029531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:22.029531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:22.029531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:22.029531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:22.029531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:22.029531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:22.029531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:16:22.029531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:16:22.029531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:16:22.030488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:16:22.030488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:22.030488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:22.030488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:22.030488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:22.030488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:22.030488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:22.030488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:22.030488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:22.030488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:22.030488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:22.030488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:22.030488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:22.030488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:22.030488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:22.030488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:22.030488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:22.031428 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:16:22.031428 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:16:22.031428 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:22.031428 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:22.031711 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0733e5a-1c85-48fd-97b2-10fcfdac84f1 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:22.031711 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0733e5a-1c85-48fd-97b2-10fcfdac84f1 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:22.031850 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 498/975] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:16:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:22.031929 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0733e5a-1c85-48fd-97b2-10fcfdac84f1 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:22.032776 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 478/976] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:16:22.036281 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f84847f-cf7d-41cf-b5b6-b09fe51a6d84 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:22.037363 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac1d15-a318-45e3-9e99-c99b11e42afa None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:22.037655 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac1d15-a318-45e3-9e99-c99b11e42afa None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:22.037937 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac1d15-a318-45e3-9e99-c99b11e42afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:22.056838 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-f4ac1d15-a318-45e3-9e99-c99b11e42afa None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:22.058648 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-f4ac1d15-a318-45e3-9e99-c99b11e42afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:22.099541 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:22.143732 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 479/977] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 07 15:16:22.147345 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f84847f-cf7d-41cf-b5b6-b09fe51a6d84 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:22.162738 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac1d15-a318-45e3-9e99-c99b11e42afa None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:22.188742 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb76ea55-ee33-4a06-b690-83318c675b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:22.189548 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76ea55-ee33-4a06-b690-83318c675b55 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:22.189841 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76ea55-ee33-4a06-b690-83318c675b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:22.190417 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76ea55-ee33-4a06-b690-83318c675b55 None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:22.191424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb76ea55-ee33-4a06-b690-83318c675b55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:22.191757 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb76ea55-ee33-4a06-b690-83318c675b55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:22.193575 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb76ea55-ee33-4a06-b690-83318c675b55 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:22.205657 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 499/978] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:22.211015 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb76ea55-ee33-4a06-b690-83318c675b55 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:22.212913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:22.268399 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac1d15-a318-45e3-9e99-c99b11e42afa None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:22.304621 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1cb5e4-2ad4-4e8b-91f3-c92861d34531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:22.305448 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1cb5e4-2ad4-4e8b-91f3-c92861d34531 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:22.305711 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1cb5e4-2ad4-4e8b-91f3-c92861d34531 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:22.305989 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1cb5e4-2ad4-4e8b-91f3-c92861d34531 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:22.306653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1cb5e4-2ad4-4e8b-91f3-c92861d34531 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:22.308050 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1cb5e4-2ad4-4e8b-91f3-c92861d34531 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:22.318706 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb76ea55-ee33-4a06-b690-83318c675b55 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:22.321818 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-3f1cb5e4-2ad4-4e8b-91f3-c92861d34531 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 07 15:16:22.321818 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:22.321818 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 15:16:22.321818 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 15:16:22.321818 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 15:16:22.321818 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 15:16:22.321818 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 15:16:22.321818 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:16:22.321818 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 07 15:16:22.321818 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:22.321818 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 15:16:22.321818 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:22.321818 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:22.321818 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:22.321818 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:22.321818 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:22.321818 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:22.322864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:22.322864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:22.322864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:22.322864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:22.322864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:22.322864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:22.322864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 15:16:22.322864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 15:16:22.322864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 15:16:22.322864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 15:16:22.322864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:22.322864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:22.322864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:22.322864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:22.322864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:22.322864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:22.323782 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:22.323782 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:22.323782 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:22.323782 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:22.323782 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:22.323782 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:22.323782 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:22.323782 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:22.323782 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:22.323782 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:22.323782 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 15:16:22.323782 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:16:22.323782 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 07 15:16:22.323782 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:22.323782 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 480/979] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Jul 7 15:16:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:22.331164 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1cb5e4-2ad4-4e8b-91f3-c92861d34531 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:22.344583 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:22.360313 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc8ab0-57b4-4a98-ac20-3482cbb88b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:22.361109 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc8ab0-57b4-4a98-ac20-3482cbb88b04 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:22.361393 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc8ab0-57b4-4a98-ac20-3482cbb88b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:22.361663 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc8ab0-57b4-4a98-ac20-3482cbb88b04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:22.368530 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cc8ab0-57b4-4a98-ac20-3482cbb88b04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:22.369977 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cc8ab0-57b4-4a98-ac20-3482cbb88b04 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:22.385636 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-25cc8ab0-57b4-4a98-ac20-3482cbb88b04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:16:22.385636 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:22.385636 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:22.385636 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:22.385636 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:22.385636 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:22.385636 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:22.385636 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:22.385636 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:22.385636 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:22.385636 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:22.385636 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:22.385636 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 15:16:22.385636 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 15:16:22.385636 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 15:16:22.386584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 15:16:22.386584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:22.386584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:22.386584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:22.386584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:22.386584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:22.386584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:22.386584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:22.386584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:22.386584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:22.386584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:22.386584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:22.386584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:22.386584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:22.386584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:22.386584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:22.386584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:22.387503 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 15:16:22.387503 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 15:16:22.387503 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 15:16:22.387503 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 15:16:22.387503 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 15:16:22.387503 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 15:16:22.387503 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:16:22.387503 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:22.387503 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 500/980] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:16:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:16:22.393041 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc8ab0-57b4-4a98-ac20-3482cbb88b04 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:22.440446 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1cb5e4-2ad4-4e8b-91f3-c92861d34531 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:22.453401 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:22.473719 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b00a92b-b16a-45a5-8c1c-1a4c8e392ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:22.474380 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00a92b-b16a-45a5-8c1c-1a4c8e392ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:22.474671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00a92b-b16a-45a5-8c1c-1a4c8e392ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:22.474950 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00a92b-b16a-45a5-8c1c-1a4c8e392ab1 None admin] PATH_INFO: `/v3/services` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:22.475684 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b00a92b-b16a-45a5-8c1c-1a4c8e392ab1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:22.476000 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b00a92b-b16a-45a5-8c1c-1a4c8e392ab1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:22.477374 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b00a92b-b16a-45a5-8c1c-1a4c8e392ab1 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:22.483121 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 481/981] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Jul 7 15:16:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:22.484652 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6e7fd-ad1e-48fc-af35-c2a9d703e393 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:22.485129 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6e7fd-ad1e-48fc-af35-c2a9d703e393 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:22.485695 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6e7fd-ad1e-48fc-af35-c2a9d703e393 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:22.487076 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 482/982] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:16:22.488603 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f6e7fd-ad1e-48fc-af35-c2a9d703e393 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:22.498230 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc8ab0-57b4-4a98-ac20-3482cbb88b04 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:22.531425 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4ce6a-6543-47a9-a4a8-9fff795b9536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:22.532038 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4ce6a-6543-47a9-a4a8-9fff795b9536 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:22.532319 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4ce6a-6543-47a9-a4a8-9fff795b9536 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:22.532593 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4ce6a-6543-47a9-a4a8-9fff795b9536 None admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:22.533374 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4ce6a-6543-47a9-a4a8-9fff795b9536 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:22.533677 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4ce6a-6543-47a9-a4a8-9fff795b9536 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:22.535093 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4ce6a-6543-47a9-a4a8-9fff795b9536 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:22.546919 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 501/983] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Jul 7 15:16:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:22.548206 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63acbd-8712-4b2a-9e55-58c585e32773 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:22.548495 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63acbd-8712-4b2a-9e55-58c585e32773 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:22.548776 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63acbd-8712-4b2a-9e55-58c585e32773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:22.564945 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-2e63acbd-8712-4b2a-9e55-58c585e32773 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:22.566133 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-2e63acbd-8712-4b2a-9e55-58c585e32773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:22.590707 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f6e7fd-ad1e-48fc-af35-c2a9d703e393 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:22.599113 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:22.626068 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c3f45-db2b-42c9-a340-674b7bd57e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:22.626801 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c3f45-db2b-42c9-a340-674b7bd57e3e None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:22.627156 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c3f45-db2b-42c9-a340-674b7bd57e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:22.627503 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c3f45-db2b-42c9-a340-674b7bd57e3e None admin] PATH_INFO: `/v3/services` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:22.628305 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c3f45-db2b-42c9-a340-674b7bd57e3e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:22.628613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c3f45-db2b-42c9-a340-674b7bd57e3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:22.629966 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c3f45-db2b-42c9-a340-674b7bd57e3e None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:22.632020 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 502/984] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:16:22.632813 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e63acbd-8712-4b2a-9e55-58c585e32773 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:22.634937 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 483/985] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Jul 7 15:16:22 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:22.645184 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c3f45-db2b-42c9-a340-674b7bd57e3e None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:22.695738 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:22.733810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e63acbd-8712-4b2a-9e55-58c585e32773 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:22.753229 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c3f45-db2b-42c9-a340-674b7bd57e3e None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:22.767617 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6768c-b3dc-4cc0-b3bb-b7e360badef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:22.768280 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6768c-b3dc-4cc0-b3bb-b7e360badef2 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:22.768563 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6768c-b3dc-4cc0-b3bb-b7e360badef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:22.768849 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6768c-b3dc-4cc0-b3bb-b7e360badef2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:22.769682 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a6768c-b3dc-4cc0-b3bb-b7e360badef2 None admin] RBAC: Adding query filter params (role.id=d5899b9e6b9940aa9837652e51a23908, scope.project.id=c5260c7fd3614e9e93b167c86a94260e, user.id=cd3c04f39d774998b66e6ce920925691) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:22.769989 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a6768c-b3dc-4cc0-b3bb-b7e360badef2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:22.771416 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a6768c-b3dc-4cc0-b3bb-b7e360badef2 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:22.777153 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 503/986] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:16:22 2022] GET /identity/v3/role_assignments?user.id=cd3c04f39d774998b66e6ce920925691&scope.project.id=c5260c7fd3614e9e93b167c86a94260e&role.id=d5899b9e6b9940aa9837652e51a23908 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:22.777971 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6768c-b3dc-4cc0-b3bb-b7e360badef2 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:22.787625 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ef14c-859b-4a2f-86d9-5f8aeb05473b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:22.788230 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ef14c-859b-4a2f-86d9-5f8aeb05473b None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:22.788514 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ef14c-859b-4a2f-86d9-5f8aeb05473b None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:22.788792 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ef14c-859b-4a2f-86d9-5f8aeb05473b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:22.789622 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ef14c-859b-4a2f-86d9-5f8aeb05473b None admin] RBAC: Adding query filter params (interface=admin, service_id=2ce7dc3ac1a44b938f915b5c261b4c6e, region_id=RegionOne) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:22.789942 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ef14c-859b-4a2f-86d9-5f8aeb05473b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:22.791344 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ef14c-859b-4a2f-86d9-5f8aeb05473b None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:22.797842 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 484/987] 10.222.0.25 () {56 vars in 1249 bytes} [Thu Jul 7 15:16:22 2022] GET /identity/v3/endpoints?service_id=2ce7dc3ac1a44b938f915b5c261b4c6e&interface=admin®ion_id=RegionOne => generated 193 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:22.803504 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ef14c-859b-4a2f-86d9-5f8aeb05473b None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:22.833246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:22.914356 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6768c-b3dc-4cc0-b3bb-b7e360badef2 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:22.918458 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ef14c-859b-4a2f-86d9-5f8aeb05473b None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:22.922865 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:22.960610 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee20287-d26d-46ae-b5f6-8f7525126d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:22.961286 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee20287-d26d-46ae-b5f6-8f7525126d19 None admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:22.961824 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee20287-d26d-46ae-b5f6-8f7525126d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:22.961904 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6659f5-a4e4-4edf-a38d-8ee86abce95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:22.962003 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee20287-d26d-46ae-b5f6-8f7525126d19 None admin] PATH_INFO: `/v3/services` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:22.962276 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6659f5-a4e4-4edf-a38d-8ee86abce95e None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:22.962548 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee20287-d26d-46ae-b5f6-8f7525126d19 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:22.962647 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6659f5-a4e4-4edf-a38d-8ee86abce95e None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:22.962852 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6659f5-a4e4-4edf-a38d-8ee86abce95e None admin] PATH_INFO: `/v3/services` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:22.963514 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6659f5-a4e4-4edf-a38d-8ee86abce95e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:22.964049 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee20287-d26d-46ae-b5f6-8f7525126d19 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:22.964924 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6659f5-a4e4-4edf-a38d-8ee86abce95e None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:22.973087 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 485/988] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Jul 7 15:16:22 2022] GET /identity/v3/services => generated 1497 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:16:23.025842 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 504/989] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Jul 7 15:16:22 2022] POST /identity/v3/services => generated 242 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:16:24.829734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2597a9-a31f-4623-bd1f-5411591c7016 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:24.830307 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2597a9-a31f-4623-bd1f-5411591c7016 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:24.830307 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2597a9-a31f-4623-bd1f-5411591c7016 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:24.831712 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 486/990] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:16:24.836816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e44339-4020-44e0-9373-5dd6f6abb6f0 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:24.837209 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e44339-4020-44e0-9373-5dd6f6abb6f0 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:24.837388 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e44339-4020-44e0-9373-5dd6f6abb6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:24.854612 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-96e44339-4020-44e0-9373-5dd6f6abb6f0 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:24.855892 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-96e44339-4020-44e0-9373-5dd6f6abb6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:24.931340 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 505/991] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:24 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:16:24.954049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2597a9-a31f-4623-bd1f-5411591c7016 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:25.014915 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34882979-8d63-4c45-87cc-ea5f9181e241 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:25.015260 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34882979-8d63-4c45-87cc-ea5f9181e241 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:25.015460 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34882979-8d63-4c45-87cc-ea5f9181e241 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:25.016844 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 506/992] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:16:25.022041 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c6cf2-0421-4ec3-a9a1-e04da0763a63 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:25.022333 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c6cf2-0421-4ec3-a9a1-e04da0763a63 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:25.022608 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c6cf2-0421-4ec3-a9a1-e04da0763a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:25.039863 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-d05c6cf2-0421-4ec3-a9a1-e04da0763a63 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:25.041036 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-d05c6cf2-0421-4ec3-a9a1-e04da0763a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:25.055268 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:25.065175 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2597a9-a31f-4623-bd1f-5411591c7016 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:25.099802 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece9cf6a-1d0a-4d61-ae27-e3fb8c4df4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:25.100502 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece9cf6a-1d0a-4d61-ae27-e3fb8c4df4c4 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:25.100775 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece9cf6a-1d0a-4d61-ae27-e3fb8c4df4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:25.101043 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece9cf6a-1d0a-4d61-ae27-e3fb8c4df4c4 None admin] PATH_INFO: `/v3/services/image` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:25.101683 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece9cf6a-1d0a-4d61-ae27-e3fb8c4df4c4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:25.103021 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece9cf6a-1d0a-4d61-ae27-e3fb8c4df4c4 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:25.111407 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 507/993] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:25 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:16:25.112758 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f99d1b-4f42-45e9-b94b-c64b9f484259 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:25.113047 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f99d1b-4f42-45e9-b94b-c64b9f484259 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:25.113325 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f99d1b-4f42-45e9-b94b-c64b9f484259 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:25.114410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-ece9cf6a-1d0a-4d61-ae27-e3fb8c4df4c4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 07 15:16:25.114410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:25.114410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 15:16:25.114410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 15:16:25.114410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 15:16:25.114410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 15:16:25.114410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 15:16:25.114410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:16:25.114410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 07 15:16:25.114410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:25.114410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 15:16:25.114410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:25.114410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:25.114410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:25.114410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:25.114410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:25.114410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:25.115383 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:25.115383 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:25.115383 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:25.115383 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:25.115383 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:25.115383 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:25.115383 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 15:16:25.115383 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 15:16:25.115383 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 15:16:25.115383 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 15:16:25.115383 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:25.115383 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:25.115383 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:25.115383 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:25.115383 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:25.115383 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:25.121275 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:25.121275 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:25.121275 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:25.121275 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:25.121275 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:25.121275 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:25.121275 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:25.121275 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:25.121275 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:25.121275 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:25.121275 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 15:16:25.121275 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:16:25.121275 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 07 15:16:25.121275 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:25.121275 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 487/994] 10.222.0.25 () {56 vars in 1105 bytes} [Thu Jul 7 15:16:24 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:25.122623 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 508/995] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:16:25.122623 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f99d1b-4f42-45e9-b94b-c64b9f484259 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:25.122764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab4862-d48b-48ee-9cfa-a2ea886a8245 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:25.122764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab4862-d48b-48ee-9cfa-a2ea886a8245 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:25.122764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab4862-d48b-48ee-9cfa-a2ea886a8245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:25.140471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-adab4862-d48b-48ee-9cfa-a2ea886a8245 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:25.141643 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-adab4862-d48b-48ee-9cfa-a2ea886a8245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:25.166829 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:25.208056 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 488/996] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:25 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:16:25.208866 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab4862-d48b-48ee-9cfa-a2ea886a8245 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:25.228243 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f99d1b-4f42-45e9-b94b-c64b9f484259 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:25.262808 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea62368a-0f2d-40e9-8e61-90c160fce64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:25.263458 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62368a-0f2d-40e9-8e61-90c160fce64c None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:25.263769 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62368a-0f2d-40e9-8e61-90c160fce64c None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:25.264050 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62368a-0f2d-40e9-8e61-90c160fce64c None admin] PATH_INFO: `/v3/services` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:25.264750 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea62368a-0f2d-40e9-8e61-90c160fce64c None admin] RBAC: Adding query filter params (name=image) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:25.265063 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea62368a-0f2d-40e9-8e61-90c160fce64c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:25.266426 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea62368a-0f2d-40e9-8e61-90c160fce64c None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:25.271168 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 509/997] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:16:25 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:25.271944 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea62368a-0f2d-40e9-8e61-90c160fce64c None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:25.286860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:25.313839 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab4862-d48b-48ee-9cfa-a2ea886a8245 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:25.347249 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3612d412-9977-4322-bdaa-ed1a553d82c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:25.348333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612d412-9977-4322-bdaa-ed1a553d82c7 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:25.348610 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612d412-9977-4322-bdaa-ed1a553d82c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:25.348880 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612d412-9977-4322-bdaa-ed1a553d82c7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:25.349532 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3612d412-9977-4322-bdaa-ed1a553d82c7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:25.350944 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3612d412-9977-4322-bdaa-ed1a553d82c7 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:25.363414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-3612d412-9977-4322-bdaa-ed1a553d82c7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 07 15:16:25.363414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:25.363414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 15:16:25.363414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 15:16:25.363414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 15:16:25.363414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 15:16:25.363414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 15:16:25.363414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:16:25.363414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 07 15:16:25.363414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:25.363414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 15:16:25.363414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:25.363414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:25.363414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:25.363414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:25.363414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:25.363414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:25.364519 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:25.364519 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:25.364519 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:25.364519 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:25.364519 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:25.364519 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:25.364519 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 15:16:25.364519 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 15:16:25.364519 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 15:16:25.364519 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 15:16:25.364519 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:25.364519 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:25.364519 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:25.364519 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:25.364519 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:25.364519 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:25.365363 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:25.365363 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:25.365363 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:25.365363 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:25.365363 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:25.365363 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:25.365363 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:25.365363 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:25.365363 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:25.365363 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:25.365363 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 15:16:25.365363 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:16:25.365363 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 07 15:16:25.365363 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:25.365363 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 489/998] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Jul 7 15:16:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:25.366202 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3612d412-9977-4322-bdaa-ed1a553d82c7 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:25.380068 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea62368a-0f2d-40e9-8e61-90c160fce64c None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:25.391891 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:25.414193 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e729c2-867c-4d18-aa08-bdc814298230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:25.414980 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e729c2-867c-4d18-aa08-bdc814298230 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:25.415262 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e729c2-867c-4d18-aa08-bdc814298230 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:25.415535 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e729c2-867c-4d18-aa08-bdc814298230 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:25.416208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e729c2-867c-4d18-aa08-bdc814298230 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:25.417575 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e729c2-867c-4d18-aa08-bdc814298230 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:25.429577 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-11e729c2-867c-4d18-aa08-bdc814298230 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 07 15:16:25.429577 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:25.429577 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 15:16:25.429577 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 15:16:25.429577 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 15:16:25.429577 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 15:16:25.429577 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 15:16:25.429577 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:16:25.429577 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 07 15:16:25.429577 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:25.429577 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 15:16:25.429577 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:25.429577 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:25.429577 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:25.429577 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:25.429577 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:25.429577 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:25.430701 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:25.430701 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:25.430701 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:25.430701 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:25.430701 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:25.430701 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:25.430701 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 15:16:25.430701 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 15:16:25.430701 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 15:16:25.430701 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 15:16:25.430701 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:25.430701 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:25.430701 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:25.430701 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:25.430701 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:25.430701 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:25.431584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:25.431584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:25.431584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:25.431584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:25.431584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:25.431584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:25.431584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:25.431584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:25.431584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:25.431584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:25.431584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 15:16:25.431584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:16:25.431584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 07 15:16:25.431584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:25.431584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 510/999] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:16:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:25.432418 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e729c2-867c-4d18-aa08-bdc814298230 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:25.466952 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3612d412-9977-4322-bdaa-ed1a553d82c7 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:25.500437 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc505f45-0e91-4783-af16-83ea1bd8cd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:25.501085 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc505f45-0e91-4783-af16-83ea1bd8cd05 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:25.501369 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc505f45-0e91-4783-af16-83ea1bd8cd05 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:25.501645 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc505f45-0e91-4783-af16-83ea1bd8cd05 None admin] PATH_INFO: `/v3/services` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:25.502352 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc505f45-0e91-4783-af16-83ea1bd8cd05 None admin] RBAC: Adding query filter params (type=image) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:25.502656 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc505f45-0e91-4783-af16-83ea1bd8cd05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:25.504040 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc505f45-0e91-4783-af16-83ea1bd8cd05 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:25.508420 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 490/1000] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:16:25 2022] GET /identity/v3/services?type=image => generated 122 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:25.509180 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc505f45-0e91-4783-af16-83ea1bd8cd05 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:25.525759 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:25.552654 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e729c2-867c-4d18-aa08-bdc814298230 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:25.596566 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce97d3-508e-4261-8b0d-c4428ef0bc47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:25.597262 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce97d3-508e-4261-8b0d-c4428ef0bc47 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:25.597804 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce97d3-508e-4261-8b0d-c4428ef0bc47 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:25.598082 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce97d3-508e-4261-8b0d-c4428ef0bc47 None admin] PATH_INFO: `/v3/services` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:25.598807 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce97d3-508e-4261-8b0d-c4428ef0bc47 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:25.599114 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce97d3-508e-4261-8b0d-c4428ef0bc47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:25.600625 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce97d3-508e-4261-8b0d-c4428ef0bc47 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:25.606469 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 511/1001] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Jul 7 15:16:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:25.610578 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:25.612104 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce97d3-508e-4261-8b0d-c4428ef0bc47 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:25.621062 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc505f45-0e91-4783-af16-83ea1bd8cd05 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:25.665223 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dfe381-4288-4c40-a3dd-a244e59f2162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:25.666131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dfe381-4288-4c40-a3dd-a244e59f2162 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:25.666131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dfe381-4288-4c40-a3dd-a244e59f2162 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:25.666428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dfe381-4288-4c40-a3dd-a244e59f2162 None admin] PATH_INFO: `/v3/services` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:25.667135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dfe381-4288-4c40-a3dd-a244e59f2162 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:25.667633 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dfe381-4288-4c40-a3dd-a244e59f2162 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:25.668865 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dfe381-4288-4c40-a3dd-a244e59f2162 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:25.676028 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 491/1002] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Jul 7 15:16:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:25.682391 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dfe381-4288-4c40-a3dd-a244e59f2162 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:25.730621 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce97d3-508e-4261-8b0d-c4428ef0bc47 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:25.769796 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd60d5-475c-4149-b6c3-870b48500b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:25.770456 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd60d5-475c-4149-b6c3-870b48500b65 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:25.770739 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd60d5-475c-4149-b6c3-870b48500b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:25.771013 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd60d5-475c-4149-b6c3-870b48500b65 None admin] PATH_INFO: `/v3/services` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:25.771753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cd60d5-475c-4149-b6c3-870b48500b65 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:25.772062 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cd60d5-475c-4149-b6c3-870b48500b65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:25.773496 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cd60d5-475c-4149-b6c3-870b48500b65 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:25.779373 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 512/1003] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Jul 7 15:16:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:25.784664 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd60d5-475c-4149-b6c3-870b48500b65 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:25.788846 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dfe381-4288-4c40-a3dd-a244e59f2162 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:25.793938 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:25.828101 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e96c5-ad88-4d0b-9dba-82da56add365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:25.828893 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e96c5-ad88-4d0b-9dba-82da56add365 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:25.829181 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e96c5-ad88-4d0b-9dba-82da56add365 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:25.829644 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e96c5-ad88-4d0b-9dba-82da56add365 None admin] PATH_INFO: `/v3/services` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:25.830354 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2e96c5-ad88-4d0b-9dba-82da56add365 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:25.830893 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2e96c5-ad88-4d0b-9dba-82da56add365 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:25.832024 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2e96c5-ad88-4d0b-9dba-82da56add365 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:25.842514 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 492/1004] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Jul 7 15:16:25 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:25.843370 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e96c5-ad88-4d0b-9dba-82da56add365 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:25.863925 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:25.890439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd60d5-475c-4149-b6c3-870b48500b65 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:25.931004 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb54300-65eb-444a-b7e9-0089a4d77203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:25.931830 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb54300-65eb-444a-b7e9-0089a4d77203 None admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:25.932227 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb54300-65eb-444a-b7e9-0089a4d77203 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:25.932509 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb54300-65eb-444a-b7e9-0089a4d77203 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:25.933490 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb54300-65eb-444a-b7e9-0089a4d77203 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:25.935820 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb54300-65eb-444a-b7e9-0089a4d77203 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:25.949905 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e96c5-ad88-4d0b-9dba-82da56add365 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:25.986749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c18e144-ea8b-43e6-a58c-d598cfb01499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:25.987410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c18e144-ea8b-43e6-a58c-d598cfb01499 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:25.987730 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c18e144-ea8b-43e6-a58c-d598cfb01499 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:25.988005 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c18e144-ea8b-43e6-a58c-d598cfb01499 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:25.988799 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c18e144-ea8b-43e6-a58c-d598cfb01499 None admin] RBAC: Adding query filter params (interface=public, service_id=998296f11c3a4ecd830cf605de76e465, region_id=RegionOne) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:25.989116 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c18e144-ea8b-43e6-a58c-d598cfb01499 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:25.990471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c18e144-ea8b-43e6-a58c-d598cfb01499 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:25.995526 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 493/1005] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Jul 7 15:16:25 2022] GET /identity/v3/endpoints?service_id=998296f11c3a4ecd830cf605de76e465&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:26.001363 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c18e144-ea8b-43e6-a58c-d598cfb01499 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:26.006152 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 513/1006] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Jul 7 15:16:25 2022] POST /identity/v3/endpoints => generated 325 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:16:26.093406 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:26.112503 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c18e144-ea8b-43e6-a58c-d598cfb01499 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:26.157192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7cffe1-1eaa-400c-b83c-c926d675609b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:26.157794 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7cffe1-1eaa-400c-b83c-c926d675609b None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:26.158102 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7cffe1-1eaa-400c-b83c-c926d675609b None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:26.159190 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7cffe1-1eaa-400c-b83c-c926d675609b None admin] PATH_INFO: `/v3/services` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:26.159190 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7cffe1-1eaa-400c-b83c-c926d675609b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:26.160395 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7cffe1-1eaa-400c-b83c-c926d675609b None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:26.168343 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 494/1007] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Jul 7 15:16:26 2022] GET /identity/v3/services => generated 1727 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:16:27.537458 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae5292-0a64-4681-b575-3c4266feae32 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:27.537976 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae5292-0a64-4681-b575-3c4266feae32 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:27.537976 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae5292-0a64-4681-b575-3c4266feae32 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:27.539519 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 514/1008] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:16:27.544948 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a7008-b28b-4516-9ebd-fdea9cf5cab5 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:27.545382 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a7008-b28b-4516-9ebd-fdea9cf5cab5 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:27.545526 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a7008-b28b-4516-9ebd-fdea9cf5cab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:27.562660 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-952a7008-b28b-4516-9ebd-fdea9cf5cab5 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:27.563864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-952a7008-b28b-4516-9ebd-fdea9cf5cab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:27.631609 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 495/1009] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:27 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:16:27.653030 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ae5292-0a64-4681-b575-3c4266feae32 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:27.722899 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:27.758663 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ae5292-0a64-4681-b575-3c4266feae32 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:27.792870 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c40cc-5f3f-4957-a48a-7c3e104c8263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:27.793503 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c40cc-5f3f-4957-a48a-7c3e104c8263 None admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:27.793876 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c40cc-5f3f-4957-a48a-7c3e104c8263 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:27.794658 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c40cc-5f3f-4957-a48a-7c3e104c8263 None admin] PATH_INFO: `/v3/services` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:27.795386 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c40cc-5f3f-4957-a48a-7c3e104c8263 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:27.796781 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c40cc-5f3f-4957-a48a-7c3e104c8263 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:27.888193 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 515/1010] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Jul 7 15:16:27 2022] POST /identity/v3/services => generated 242 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:16:28.061555 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa7951-4724-4292-b314-1d7061f3005c None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:28.061851 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa7951-4724-4292-b314-1d7061f3005c None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:28.062040 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa7951-4724-4292-b314-1d7061f3005c None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:28.063460 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 496/1011] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:16:28.068642 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3f82f-dd85-486e-bde1-cbf32b0f6e0f None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:28.068937 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3f82f-dd85-486e-bde1-cbf32b0f6e0f None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:28.069214 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3f82f-dd85-486e-bde1-cbf32b0f6e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:28.086712 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-70d3f82f-dd85-486e-bde1-cbf32b0f6e0f None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:28.087949 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-70d3f82f-dd85-486e-bde1-cbf32b0f6e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:28.164951 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 516/1012] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:28 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:16:28.188171 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa7951-4724-4292-b314-1d7061f3005c None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:28.204171 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac02510-fb9c-4541-8fc2-2af3ec31e941 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:28.204463 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac02510-fb9c-4541-8fc2-2af3ec31e941 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:28.204743 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac02510-fb9c-4541-8fc2-2af3ec31e941 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:28.206097 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 517/1013] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:16:28.211051 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a578d-e12c-4359-9e21-33f2be2169fe None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:28.211334 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a578d-e12c-4359-9e21-33f2be2169fe None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:28.211638 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a578d-e12c-4359-9e21-33f2be2169fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:28.229720 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-1f3a578d-e12c-4359-9e21-33f2be2169fe None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:28.230914 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-1f3a578d-e12c-4359-9e21-33f2be2169fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:28.271442 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:28.280268 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:28.300231 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 518/1014] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:28 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 07 15:16:28.306051 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa7951-4724-4292-b314-1d7061f3005c None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:28.323138 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a578d-e12c-4359-9e21-33f2be2169fe None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:28.346218 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c18a6-d9bd-42f8-a956-4076d199f573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:28.347012 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c18a6-d9bd-42f8-a956-4076d199f573 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:28.347298 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c18a6-d9bd-42f8-a956-4076d199f573 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:28.347878 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c18a6-d9bd-42f8-a956-4076d199f573 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:28.349151 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75c18a6-d9bd-42f8-a956-4076d199f573 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:28.350586 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75c18a6-d9bd-42f8-a956-4076d199f573 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:28.352271 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 497/1015] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:16:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:28.359158 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c18a6-d9bd-42f8-a956-4076d199f573 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:28.437940 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a578d-e12c-4359-9e21-33f2be2169fe None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:28.464355 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c18a6-d9bd-42f8-a956-4076d199f573 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:28.472978 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bbf163-34f2-4369-a4ac-75843dad1aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:28.473804 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bbf163-34f2-4369-a4ac-75843dad1aa9 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:28.474091 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bbf163-34f2-4369-a4ac-75843dad1aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:28.474371 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bbf163-34f2-4369-a4ac-75843dad1aa9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:28.475021 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bbf163-34f2-4369-a4ac-75843dad1aa9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:28.476424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bbf163-34f2-4369-a4ac-75843dad1aa9 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:28.488630 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-b3bbf163-34f2-4369-a4ac-75843dad1aa9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 07 15:16:28.488630 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:28.488630 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 15:16:28.488630 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 15:16:28.488630 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 15:16:28.488630 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 15:16:28.488630 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 15:16:28.488630 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:16:28.488630 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 07 15:16:28.488630 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:28.488630 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 15:16:28.488630 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:28.488630 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:28.488630 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:28.488630 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:28.488630 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:28.488630 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:28.489707 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:28.489707 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:28.489707 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:28.489707 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:28.489707 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:28.489707 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:28.489707 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 15:16:28.489707 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 15:16:28.489707 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 15:16:28.489707 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 15:16:28.489707 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:28.489707 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:28.489707 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:28.489707 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:28.489707 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:28.489707 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:28.492628 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:28.492628 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:28.492628 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:28.492628 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:28.492628 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:28.492628 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:28.492628 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:28.492628 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:28.492628 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:28.492628 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:28.492628 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 15:16:28.492628 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:16:28.492628 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 07 15:16:28.492628 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:28.492628 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 519/1016] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:16:28 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:16:28.495596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bbf163-34f2-4369-a4ac-75843dad1aa9 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:28.502495 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd0807-e8d9-4088-8ba9-e6939e29b765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:28.503162 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd0807-e8d9-4088-8ba9-e6939e29b765 None admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:28.503444 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd0807-e8d9-4088-8ba9-e6939e29b765 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:28.503762 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd0807-e8d9-4088-8ba9-e6939e29b765 None admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:28.504472 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fd0807-e8d9-4088-8ba9-e6939e29b765 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:28.505891 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fd0807-e8d9-4088-8ba9-e6939e29b765 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:28.516464 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:28.543157 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:28.595114 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bbf163-34f2-4369-a4ac-75843dad1aa9 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:28.631596 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 498/1017] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Jul 7 15:16:28 2022] POST /identity/v3/projects => generated 311 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:16:28.639764 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ad36ba-b21a-4f4d-a517-110229885873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:28.640735 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad36ba-b21a-4f4d-a517-110229885873 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:28.641022 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad36ba-b21a-4f4d-a517-110229885873 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:28.641300 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad36ba-b21a-4f4d-a517-110229885873 None admin] PATH_INFO: `/v3/services` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:28.642042 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ad36ba-b21a-4f4d-a517-110229885873 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:28.642353 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ad36ba-b21a-4f4d-a517-110229885873 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:28.643798 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ad36ba-b21a-4f4d-a517-110229885873 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:28.650004 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 520/1018] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Jul 7 15:16:28 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:28.656363 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd0807-e8d9-4088-8ba9-e6939e29b765 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:28.786235 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd0807-e8d9-4088-8ba9-e6939e29b765 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:28.822142 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e6925b-a0da-4bae-a596-1b2b1c3423cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:28.822802 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6925b-a0da-4bae-a596-1b2b1c3423cf None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:28.823086 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6925b-a0da-4bae-a596-1b2b1c3423cf None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:28.823362 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6925b-a0da-4bae-a596-1b2b1c3423cf None admin] PATH_INFO: `/v3/services` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:28.824102 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e6925b-a0da-4bae-a596-1b2b1c3423cf None admin] RBAC: Adding query filter params (type=compute) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:28.824415 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e6925b-a0da-4bae-a596-1b2b1c3423cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:28.825805 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e6925b-a0da-4bae-a596-1b2b1c3423cf None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:28.830612 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 499/1019] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Jul 7 15:16:28 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:28.837459 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ad36ba-b21a-4f4d-a517-110229885873 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:28.939734 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ad36ba-b21a-4f4d-a517-110229885873 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:28.950675 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:28.977765 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cdbd92-d2fb-4d7a-85e7-b3f93c134686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:28.978433 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cdbd92-d2fb-4d7a-85e7-b3f93c134686 None admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:28.978711 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cdbd92-d2fb-4d7a-85e7-b3f93c134686 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:28.978985 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cdbd92-d2fb-4d7a-85e7-b3f93c134686 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:28.979765 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cdbd92-d2fb-4d7a-85e7-b3f93c134686 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:28.981129 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cdbd92-d2fb-4d7a-85e7-b3f93c134686 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:29.057551 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 521/1020] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Jul 7 15:16:28 2022] POST /identity/v3/endpoints => generated 334 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:16:29.942347 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc208c-92fe-403c-b336-39bf2a9727f4 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:29.942898 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc208c-92fe-403c-b336-39bf2a9727f4 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:29.942898 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc208c-92fe-403c-b336-39bf2a9727f4 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:29.944436 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 500/1021] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:16:29.949395 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee4ee1-b0ab-4a4d-8f84-d64afaaebf0b None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:29.949770 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee4ee1-b0ab-4a4d-8f84-d64afaaebf0b None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:29.949978 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee4ee1-b0ab-4a4d-8f84-d64afaaebf0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:29.969746 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-85ee4ee1-b0ab-4a4d-8f84-d64afaaebf0b None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:29.970934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-85ee4ee1-b0ab-4a4d-8f84-d64afaaebf0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:30.047609 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 522/1022] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:29 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:16:30.067844 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cc208c-92fe-403c-b336-39bf2a9727f4 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:30.130877 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:30.184261 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cc208c-92fe-403c-b336-39bf2a9727f4 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:30.223001 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e027cced-c83a-48ab-ae2c-1167b51306ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:30.223746 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027cced-c83a-48ab-ae2c-1167b51306ff None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:30.224031 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027cced-c83a-48ab-ae2c-1167b51306ff None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:30.224305 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027cced-c83a-48ab-ae2c-1167b51306ff None admin] PATH_INFO: `/v3/services/image` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:30.224949 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e027cced-c83a-48ab-ae2c-1167b51306ff None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:30.226302 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e027cced-c83a-48ab-ae2c-1167b51306ff None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:30.238460 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-e027cced-c83a-48ab-ae2c-1167b51306ff None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 07 15:16:30.238460 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:30.238460 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 15:16:30.238460 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 15:16:30.238460 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 15:16:30.238460 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 15:16:30.238460 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 15:16:30.238460 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:16:30.238460 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 07 15:16:30.238460 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:30.238460 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 15:16:30.238460 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:30.238460 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:30.238460 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:30.238460 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:30.238460 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:30.238460 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:30.239492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:30.239492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:30.239492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:30.239492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:30.239492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:30.239492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:30.239492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 15:16:30.239492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 15:16:30.239492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 15:16:30.239492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 15:16:30.239492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:30.239492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:30.239492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:30.239492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:30.239492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:30.239492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:30.240683 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:30.240683 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:30.240683 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:30.240683 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:30.240683 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:30.240683 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:30.240683 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:30.240683 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:30.240683 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:30.240683 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:30.240683 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 15:16:30.240683 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:16:30.240683 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 07 15:16:30.240683 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:30.241884 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 501/1023] 10.222.0.25 () {56 vars in 1105 bytes} [Thu Jul 7 15:16:30 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:30.245825 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee4ee1-b0ab-4a4d-8f84-d64afaaebf0b None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:30.342124 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:30.351421 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee4ee1-b0ab-4a4d-8f84-d64afaaebf0b None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:30.386470 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4778aa6-8344-407b-9c42-8c1eacf6bd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:30.387177 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4778aa6-8344-407b-9c42-8c1eacf6bd05 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:30.388019 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4778aa6-8344-407b-9c42-8c1eacf6bd05 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:30.388305 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4778aa6-8344-407b-9c42-8c1eacf6bd05 None admin] PATH_INFO: `/v3/services` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:30.389013 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4778aa6-8344-407b-9c42-8c1eacf6bd05 None admin] RBAC: Adding query filter params (name=image) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:30.389320 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4778aa6-8344-407b-9c42-8c1eacf6bd05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:30.390680 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4778aa6-8344-407b-9c42-8c1eacf6bd05 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:30.400888 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 523/1024] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:16:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:30.401646 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e027cced-c83a-48ab-ae2c-1167b51306ff None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:30.510986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e027cced-c83a-48ab-ae2c-1167b51306ff None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:30.531777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:30.546109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ca552-5e87-43b4-a4f6-097f559a7c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:30.546772 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ca552-5e87-43b4-a4f6-097f559a7c3f None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:30.547055 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ca552-5e87-43b4-a4f6-097f559a7c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:30.547331 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ca552-5e87-43b4-a4f6-097f559a7c3f None admin] PATH_INFO: `/v3/services` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:30.548077 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6ca552-5e87-43b4-a4f6-097f559a7c3f None admin] RBAC: Adding query filter params (type=image) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:30.548388 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6ca552-5e87-43b4-a4f6-097f559a7c3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:30.549746 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6ca552-5e87-43b4-a4f6-097f559a7c3f None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:30.559268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 502/1025] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:16:30 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:30.560387 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4778aa6-8344-407b-9c42-8c1eacf6bd05 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:30.661113 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4778aa6-8344-407b-9c42-8c1eacf6bd05 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:30.694394 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23453d44-9bc3-4e71-9d3f-67abede82b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:30.695055 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23453d44-9bc3-4e71-9d3f-67abede82b7a None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:30.695336 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23453d44-9bc3-4e71-9d3f-67abede82b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:30.695649 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23453d44-9bc3-4e71-9d3f-67abede82b7a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:30.696438 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23453d44-9bc3-4e71-9d3f-67abede82b7a None admin] RBAC: Adding query filter params (interface=public, service_id=025b6a30b22c4338a135649ae5060422, region_id=RegionOne) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:30.696749 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23453d44-9bc3-4e71-9d3f-67abede82b7a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:30.698105 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23453d44-9bc3-4e71-9d3f-67abede82b7a None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:30.708089 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 524/1026] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Jul 7 15:16:30 2022] GET /identity/v3/endpoints?service_id=025b6a30b22c4338a135649ae5060422&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:30.709237 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ca552-5e87-43b4-a4f6-097f559a7c3f None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:30.755266 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa66226-b90a-425f-b162-6777014f19f3 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:30.755629 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa66226-b90a-425f-b162-6777014f19f3 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:30.755848 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa66226-b90a-425f-b162-6777014f19f3 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:30.757460 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 525/1027] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:16:30.763491 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f4756-f0f5-412a-a9c0-b40414d1237a None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:30.763491 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f4756-f0f5-412a-a9c0-b40414d1237a None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:30.763745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f4756-f0f5-412a-a9c0-b40414d1237a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:30.791425 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-123f4756-f0f5-412a-a9c0-b40414d1237a None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:30.792648 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-123f4756-f0f5-412a-a9c0-b40414d1237a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:30.808897 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:30.826963 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ca552-5e87-43b4-a4f6-097f559a7c3f None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:30.862371 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f2beca-4308-4902-b87d-8930fbec547d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:30.863032 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f2beca-4308-4902-b87d-8930fbec547d None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:30.863312 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f2beca-4308-4902-b87d-8930fbec547d None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:30.863591 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f2beca-4308-4902-b87d-8930fbec547d None admin] PATH_INFO: `/v3/services` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:30.864283 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f2beca-4308-4902-b87d-8930fbec547d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:30.865630 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f2beca-4308-4902-b87d-8930fbec547d None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:30.867679 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 526/1028] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:30 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 07 15:16:30.880725 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 503/1029] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Jul 7 15:16:30 2022] GET /identity/v3/services => generated 1957 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:16:30.896320 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123f4756-f0f5-412a-a9c0-b40414d1237a None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:31.016592 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123f4756-f0f5-412a-a9c0-b40414d1237a None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:31.055264 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0b216f-375e-4bc7-b21c-1d2d5b526cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:31.056092 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b216f-375e-4bc7-b21c-1d2d5b526cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:31.056375 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b216f-375e-4bc7-b21c-1d2d5b526cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:31.056649 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b216f-375e-4bc7-b21c-1d2d5b526cb4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:31.057821 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0b216f-375e-4bc7-b21c-1d2d5b526cb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:31.059852 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0b216f-375e-4bc7-b21c-1d2d5b526cb4 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:31.062355 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 527/1030] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:16:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:31.067091 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f2beca-4308-4902-b87d-8930fbec547d None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:31.120719 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:31.173037 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f2beca-4308-4902-b87d-8930fbec547d None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:31.194933 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d2807a-111a-4b4c-8b44-e082ddb78cbe None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:31.195218 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d2807a-111a-4b4c-8b44-e082ddb78cbe None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:31.195495 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d2807a-111a-4b4c-8b44-e082ddb78cbe None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:31.196884 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 528/1031] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:16:31.201963 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae60bb-518d-46c8-8fce-f7c5d3c05f97 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:31.202249 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae60bb-518d-46c8-8fce-f7c5d3c05f97 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:31.202530 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae60bb-518d-46c8-8fce-f7c5d3c05f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:31.212828 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e7b15-fa5c-4d4f-b255-6ebe1fac2d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:31.213499 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e7b15-fa5c-4d4f-b255-6ebe1fac2d8e None admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:31.213781 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e7b15-fa5c-4d4f-b255-6ebe1fac2d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:31.214054 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e7b15-fa5c-4d4f-b255-6ebe1fac2d8e None admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:31.214768 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558e7b15-fa5c-4d4f-b255-6ebe1fac2d8e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:31.216246 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558e7b15-fa5c-4d4f-b255-6ebe1fac2d8e None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:31.220833 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-9bae60bb-518d-46c8-8fce-f7c5d3c05f97 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:31.222017 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-9bae60bb-518d-46c8-8fce-f7c5d3c05f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:31.239090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:31.291387 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 529/1032] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:31 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:16:31.303959 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 504/1033] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Jul 7 15:16:31 2022] POST /identity/v3/users => generated 285 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:16:31.315907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bae60bb-518d-46c8-8fce-f7c5d3c05f97 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:31.434250 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bae60bb-518d-46c8-8fce-f7c5d3c05f97 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:31.470945 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b9a88-09d2-4dbd-a5c3-f1bd17e61343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:31.471776 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b9a88-09d2-4dbd-a5c3-f1bd17e61343 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:31.472058 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b9a88-09d2-4dbd-a5c3-f1bd17e61343 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:31.472331 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b9a88-09d2-4dbd-a5c3-f1bd17e61343 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:31.483353 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88b9a88-09d2-4dbd-a5c3-f1bd17e61343 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:16:31.491517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-b88b9a88-09d2-4dbd-a5c3-f1bd17e61343 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 07 15:16:31.491517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:31.491517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:31.491517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:31.491517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:31.491517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:31.491517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:31.491517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:31.491517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:31.491517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:31.491517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:31.491517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:31.491517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:16:31.491517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:16:31.491517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:16:31.491517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:16:31.492597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:16:31.492597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:16:31.492597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:16:31.492597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:16:31.492597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:31.492597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:31.492597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:31.492597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:31.492597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:31.492597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:31.492597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:31.492597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:31.492597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:31.492597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:31.492597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:31.492597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:31.493471 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:31.493471 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:31.493471 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:31.493471 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:31.493471 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:16:31.493471 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:16:31.493471 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:16:31.493471 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:16:31.493471 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:16:31.493471 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:16:31.493471 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 07 15:16:31.493471 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:31.493471 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 530/1034] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Jul 7 15:16:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:31.498650 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e7b15-fa5c-4d4f-b255-6ebe1fac2d8e None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:31.603445 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e7b15-fa5c-4d4f-b255-6ebe1fac2d8e None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:31.607162 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:31.637347 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd8ba26-2e65-415c-b66c-1666c5208303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:31.638016 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8ba26-2e65-415c-b66c-1666c5208303 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:31.638326 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8ba26-2e65-415c-b66c-1666c5208303 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:31.638566 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8ba26-2e65-415c-b66c-1666c5208303 None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:31.639314 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd8ba26-2e65-415c-b66c-1666c5208303 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:31.639643 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd8ba26-2e65-415c-b66c-1666c5208303 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:31.640993 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd8ba26-2e65-415c-b66c-1666c5208303 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:31.650515 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 505/1035] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Jul 7 15:16:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:31.656980 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b9a88-09d2-4dbd-a5c3-f1bd17e61343 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:31.668637 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:31.757039 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b9a88-09d2-4dbd-a5c3-f1bd17e61343 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:31.789700 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be5df9a-823c-4e19-96a6-96e5ee29a15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:31.790484 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be5df9a-823c-4e19-96a6-96e5ee29a15d None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:31.790777 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be5df9a-823c-4e19-96a6-96e5ee29a15d None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:31.791029 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be5df9a-823c-4e19-96a6-96e5ee29a15d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:31.802048 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be5df9a-823c-4e19-96a6-96e5ee29a15d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:31.803482 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be5df9a-823c-4e19-96a6-96e5ee29a15d None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:31.820077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-8be5df9a-823c-4e19-96a6-96e5ee29a15d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:31.820077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:31.820077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:31.820077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:31.820077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:31.820077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:31.820077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:31.820077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:31.820077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:31.820077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:31.820077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:31.820077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:31.820077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:16:31.820077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:16:31.820077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:16:31.821007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:16:31.821007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:31.821007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:31.821007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:31.821007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:31.821007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:31.821007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:31.821007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:31.821007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:31.821007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:31.821007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:31.821007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:31.821007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:31.821007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:31.821007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:31.821007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:31.821007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:31.821921 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:16:31.821921 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:16:31.821921 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:31.821921 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:31.821921 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 531/1036] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:16:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:31.828320 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd8ba26-2e65-415c-b66c-1666c5208303 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:31.935809 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd8ba26-2e65-415c-b66c-1666c5208303 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:31.976502 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982e35c-0486-493f-96ab-636378ae77a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:31.977170 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982e35c-0486-493f-96ab-636378ae77a2 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:31.977452 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982e35c-0486-493f-96ab-636378ae77a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:31.977727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982e35c-0486-493f-96ab-636378ae77a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:31.978452 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982e35c-0486-493f-96ab-636378ae77a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:31.978760 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982e35c-0486-493f-96ab-636378ae77a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:31.980143 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982e35c-0486-493f-96ab-636378ae77a2 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:31.997118 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 506/1037] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:31.998479 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be5df9a-823c-4e19-96a6-96e5ee29a15d None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:32.062990 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:32.105009 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be5df9a-823c-4e19-96a6-96e5ee29a15d None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:32.139043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdd833e-0c64-4288-91e5-6b61b7458da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:32.139852 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd833e-0c64-4288-91e5-6b61b7458da8 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:32.140135 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd833e-0c64-4288-91e5-6b61b7458da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:32.140409 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd833e-0c64-4288-91e5-6b61b7458da8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:32.147962 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdd833e-0c64-4288-91e5-6b61b7458da8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:32.149386 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdd833e-0c64-4288-91e5-6b61b7458da8 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:32.163602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-abdd833e-0c64-4288-91e5-6b61b7458da8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 07 15:16:32.163602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:32.163602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:32.163602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:32.163602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:32.163602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:32.163602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:32.163602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:32.163602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:32.163602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:32.163602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:32.163602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:32.163602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 15:16:32.163602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 15:16:32.163602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 15:16:32.163602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 15:16:32.164684 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:32.164684 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:32.164684 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 15:16:32.164684 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:16:32.164684 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 15:16:32.164684 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:16:32.164684 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:32.164684 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:32.164684 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:32.164684 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:32.164684 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:32.164684 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:32.164684 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:32.164684 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:32.164684 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:32.164684 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:32.164684 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:32.164684 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:32.170480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:32.170480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:32.170480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 15:16:32.170480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 15:16:32.170480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 15:16:32.170480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 15:16:32.170480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 15:16:32.170480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 15:16:32.170480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 07 15:16:32.170480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:32.170480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 532/1038] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Jul 7 15:16:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:16:32.171104 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982e35c-0486-493f-96ab-636378ae77a2 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:32.202035 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:32.275051 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982e35c-0486-493f-96ab-636378ae77a2 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:32.308435 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e4a012-c6ee-428b-b6f7-da3efca22dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:32.309099 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4a012-c6ee-428b-b6f7-da3efca22dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:32.309913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4a012-c6ee-428b-b6f7-da3efca22dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:32.310188 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4a012-c6ee-428b-b6f7-da3efca22dc1 None admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:32.311019 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e4a012-c6ee-428b-b6f7-da3efca22dc1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:32.311329 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e4a012-c6ee-428b-b6f7-da3efca22dc1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:32.312847 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e4a012-c6ee-428b-b6f7-da3efca22dc1 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:32.330400 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 507/1039] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Jul 7 15:16:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:32.335796 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdd833e-0c64-4288-91e5-6b61b7458da8 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:32.437695 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdd833e-0c64-4288-91e5-6b61b7458da8 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:32.458426 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:32.472975 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784c6c9-38b4-4451-a8cf-fd29fd7db5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:32.473629 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784c6c9-38b4-4451-a8cf-fd29fd7db5c2 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:32.473913 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784c6c9-38b4-4451-a8cf-fd29fd7db5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:32.474188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784c6c9-38b4-4451-a8cf-fd29fd7db5c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:32.544393 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6784c6c9-38b4-4451-a8cf-fd29fd7db5c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:32.545841 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6784c6c9-38b4-4451-a8cf-fd29fd7db5c2 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:32.644594 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 533/1040] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:16:32 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 07 15:16:32.650063 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e4a012-c6ee-428b-b6f7-da3efca22dc1 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:32.734811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:32.751520 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e4a012-c6ee-428b-b6f7-da3efca22dc1 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:32.785306 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1184b5af-384e-4e75-8743-ab2ef62bd4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:32.786092 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184b5af-384e-4e75-8743-ab2ef62bd4b7 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:32.786377 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184b5af-384e-4e75-8743-ab2ef62bd4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:32.786655 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184b5af-384e-4e75-8743-ab2ef62bd4b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:32.797539 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1184b5af-384e-4e75-8743-ab2ef62bd4b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:32.798984 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1184b5af-384e-4e75-8743-ab2ef62bd4b7 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:32.817109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-1184b5af-384e-4e75-8743-ab2ef62bd4b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:32.817109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:32.817109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:32.817109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:32.817109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:32.817109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:32.817109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:32.817109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:32.817109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:32.817109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:32.817109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:32.817109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:32.817109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:16:32.817109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:16:32.817109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:16:32.818061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:16:32.818061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:32.818061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:32.818061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:32.818061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:32.818061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:32.818061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:32.818061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:32.818061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:32.818061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:32.818061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:32.818061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:32.818061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:32.818061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:32.818061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:32.818061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:32.818061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:32.818995 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:16:32.818995 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:16:32.818995 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:32.818995 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:32.819812 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 508/1041] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:16:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:32.823959 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784c6c9-38b4-4451-a8cf-fd29fd7db5c2 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:32.846230 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:32.924764 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784c6c9-38b4-4451-a8cf-fd29fd7db5c2 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:32.942921 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc3ce8-537d-438a-ab66-eba4a2065e21 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:32.943236 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc3ce8-537d-438a-ab66-eba4a2065e21 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:32.943518 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc3ce8-537d-438a-ab66-eba4a2065e21 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:32.945056 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 509/1042] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:16:32.950262 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315b6c2-f8f5-4437-b9bf-6e2b1a684d30 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:32.953652 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315b6c2-f8f5-4437-b9bf-6e2b1a684d30 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:32.953652 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315b6c2-f8f5-4437-b9bf-6e2b1a684d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:32.965232 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8628b0-2e16-4624-9eed-4d4949fc73d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:32.966300 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8628b0-2e16-4624-9eed-4d4949fc73d8 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:32.966583 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8628b0-2e16-4624-9eed-4d4949fc73d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:32.966852 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8628b0-2e16-4624-9eed-4d4949fc73d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:32.967602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8628b0-2e16-4624-9eed-4d4949fc73d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:32.967914 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8628b0-2e16-4624-9eed-4d4949fc73d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:32.969291 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8628b0-2e16-4624-9eed-4d4949fc73d8 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:32.969803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-c315b6c2-f8f5-4437-b9bf-6e2b1a684d30 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:32.970997 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-c315b6c2-f8f5-4437-b9bf-6e2b1a684d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:32.984365 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 534/1043] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:16:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:32.989970 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8628b0-2e16-4624-9eed-4d4949fc73d8 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:33.051599 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 510/1044] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:32 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:16:33.069948 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c315b6c2-f8f5-4437-b9bf-6e2b1a684d30 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:33.082609 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:33.103369 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8628b0-2e16-4624-9eed-4d4949fc73d8 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:33.139260 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:33.140568 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d72da-7dc1-4a17-8ede-00c00f85ed3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:33.141358 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d72da-7dc1-4a17-8ede-00c00f85ed3f None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:33.141638 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d72da-7dc1-4a17-8ede-00c00f85ed3f None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:33.141910 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d72da-7dc1-4a17-8ede-00c00f85ed3f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:33.148635 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d72da-7dc1-4a17-8ede-00c00f85ed3f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:33.150084 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d72da-7dc1-4a17-8ede-00c00f85ed3f None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:33.164934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-6d2d72da-7dc1-4a17-8ede-00c00f85ed3f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:16:33.164934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:33.164934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:33.164934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:33.164934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:33.164934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:33.164934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:33.164934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:33.164934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:33.164934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:33.164934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:33.164934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:33.164934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 15:16:33.164934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 15:16:33.164934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 15:16:33.165974 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 15:16:33.165974 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:33.165974 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:33.165974 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:33.165974 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:33.165974 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:33.165974 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:33.165974 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:33.165974 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:33.165974 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:33.165974 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:33.165974 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:33.165974 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:33.165974 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:33.165974 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:33.165974 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:33.165974 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:33.172338 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 15:16:33.172338 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 15:16:33.172338 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 15:16:33.172338 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 15:16:33.172338 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 15:16:33.172338 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 15:16:33.172338 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:16:33.172338 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:33.172338 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 535/1045] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:16:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:16:33.172840 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d72da-7dc1-4a17-8ede-00c00f85ed3f None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:33.181873 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c315b6c2-f8f5-4437-b9bf-6e2b1a684d30 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:33.216856 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7935cb2f-5bfd-4871-818e-19dc0ce1f5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:33.217636 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7935cb2f-5bfd-4871-818e-19dc0ce1f5c6 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:33.217919 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7935cb2f-5bfd-4871-818e-19dc0ce1f5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:33.218195 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7935cb2f-5bfd-4871-818e-19dc0ce1f5c6 None admin] PATH_INFO: `/v3/services/image` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:33.218846 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7935cb2f-5bfd-4871-818e-19dc0ce1f5c6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:33.220240 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7935cb2f-5bfd-4871-818e-19dc0ce1f5c6 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:33.232482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-7935cb2f-5bfd-4871-818e-19dc0ce1f5c6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 07 15:16:33.232482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:33.232482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 15:16:33.232482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 15:16:33.232482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 15:16:33.232482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 15:16:33.232482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 15:16:33.232482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:16:33.232482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 07 15:16:33.232482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:33.232482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 15:16:33.232482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:33.232482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:33.232482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:33.232482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:33.232482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:33.232482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:33.233485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:33.233485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:33.233485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:33.233485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:33.233485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:33.233485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:33.233485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 15:16:33.233485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 15:16:33.233485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 15:16:33.233485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 15:16:33.233485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:33.233485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:33.233485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:33.233485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:33.233485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:33.233485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:33.234343 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:33.234343 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:33.234343 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:33.234343 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:33.234343 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:33.234343 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:33.234343 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:33.234343 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:33.234343 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:33.234343 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:33.234343 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 15:16:33.234343 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:16:33.234343 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 07 15:16:33.234343 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:33.234343 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 511/1046] 10.222.0.25 () {56 vars in 1105 bytes} [Thu Jul 7 15:16:33 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:16:33.239436 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7935cb2f-5bfd-4871-818e-19dc0ce1f5c6 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:33.276471 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d72da-7dc1-4a17-8ede-00c00f85ed3f None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:33.312383 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068aa320-ccc3-4d66-babb-437241493c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:33.313040 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068aa320-ccc3-4d66-babb-437241493c6c None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:33.313318 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068aa320-ccc3-4d66-babb-437241493c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:33.313587 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068aa320-ccc3-4d66-babb-437241493c6c None admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:33.314366 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068aa320-ccc3-4d66-babb-437241493c6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:33.314673 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068aa320-ccc3-4d66-babb-437241493c6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:33.316127 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068aa320-ccc3-4d66-babb-437241493c6c None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:33.324989 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:33.329992 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 536/1047] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Jul 7 15:16:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:33.335372 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068aa320-ccc3-4d66-babb-437241493c6c None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:33.348631 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7935cb2f-5bfd-4871-818e-19dc0ce1f5c6 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:33.383110 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:33.389951 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c06915-2d32-4c94-83dc-a32e735fc498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:33.390818 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c06915-2d32-4c94-83dc-a32e735fc498 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:33.391169 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c06915-2d32-4c94-83dc-a32e735fc498 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:33.391587 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c06915-2d32-4c94-83dc-a32e735fc498 None admin] PATH_INFO: `/v3/services` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:33.392416 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c06915-2d32-4c94-83dc-a32e735fc498 None admin] RBAC: Adding query filter params (name=image) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:33.396581 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c06915-2d32-4c94-83dc-a32e735fc498 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:33.396581 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c06915-2d32-4c94-83dc-a32e735fc498 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:33.405022 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 512/1048] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:16:33 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:33.406018 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c06915-2d32-4c94-83dc-a32e735fc498 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:33.442366 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068aa320-ccc3-4d66-babb-437241493c6c None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:33.475722 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977fc57-1f7a-44cc-8cd2-e071d7647b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:33.476365 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977fc57-1f7a-44cc-8cd2-e071d7647b90 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:33.476644 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977fc57-1f7a-44cc-8cd2-e071d7647b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:33.476918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977fc57-1f7a-44cc-8cd2-e071d7647b90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:33.477734 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f977fc57-1f7a-44cc-8cd2-e071d7647b90 None admin] RBAC: Adding query filter params (role.id=d5899b9e6b9940aa9837652e51a23908, scope.project.id=c5260c7fd3614e9e93b167c86a94260e, user.id=de1632555cb345f2b9213da1925c02f4) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:33.478041 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f977fc57-1f7a-44cc-8cd2-e071d7647b90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:33.479446 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f977fc57-1f7a-44cc-8cd2-e071d7647b90 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:33.484870 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 537/1049] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:16:33 2022] GET /identity/v3/role_assignments?user.id=de1632555cb345f2b9213da1925c02f4&scope.project.id=c5260c7fd3614e9e93b167c86a94260e&role.id=d5899b9e6b9940aa9837652e51a23908 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:33.486090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640981c4-4028-45e4-9264-e9bc8a66c718 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:33.486375 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640981c4-4028-45e4-9264-e9bc8a66c718 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:33.486652 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640981c4-4028-45e4-9264-e9bc8a66c718 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:33.488061 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 538/1050] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:16:33.493509 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f7f68-4173-4564-a821-be3edf3e98eb None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:33.493797 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f7f68-4173-4564-a821-be3edf3e98eb None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:33.494072 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f7f68-4173-4564-a821-be3edf3e98eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:33.510514 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c06915-2d32-4c94-83dc-a32e735fc498 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:33.514122 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-515f7f68-4173-4564-a821-be3edf3e98eb None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:33.515343 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-515f7f68-4173-4564-a821-be3edf3e98eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:33.548117 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaae4f8-cbf2-4b1f-a6d4-8610bb26d123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:33.548787 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaae4f8-cbf2-4b1f-a6d4-8610bb26d123 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:33.549131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaae4f8-cbf2-4b1f-a6d4-8610bb26d123 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:33.549472 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaae4f8-cbf2-4b1f-a6d4-8610bb26d123 None admin] PATH_INFO: `/v3/services` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:33.550256 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaae4f8-cbf2-4b1f-a6d4-8610bb26d123 None admin] RBAC: Adding query filter params (type=image) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:33.550629 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaae4f8-cbf2-4b1f-a6d4-8610bb26d123 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:33.552081 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaae4f8-cbf2-4b1f-a6d4-8610bb26d123 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:33.562178 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 513/1051] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:16:33 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:33.563263 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaae4f8-cbf2-4b1f-a6d4-8610bb26d123 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:33.591879 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:33.601762 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 539/1052] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:33 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 07 15:16:33.625265 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f7f68-4173-4564-a821-be3edf3e98eb None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:33.628320 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:33.672374 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaae4f8-cbf2-4b1f-a6d4-8610bb26d123 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:33.707364 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1bfab-d8f3-4e79-8b1d-9eec5ca83b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:33.708174 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1bfab-d8f3-4e79-8b1d-9eec5ca83b95 None admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:33.708530 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1bfab-d8f3-4e79-8b1d-9eec5ca83b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:33.708807 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1bfab-d8f3-4e79-8b1d-9eec5ca83b95 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:33.709538 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea1bfab-d8f3-4e79-8b1d-9eec5ca83b95 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:33.710889 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea1bfab-d8f3-4e79-8b1d-9eec5ca83b95 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:33.742704 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f7f68-4173-4564-a821-be3edf3e98eb None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:33.776679 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258117c4-1f0b-4fe7-8aad-5061640477f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:33.777469 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258117c4-1f0b-4fe7-8aad-5061640477f2 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:33.784548 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258117c4-1f0b-4fe7-8aad-5061640477f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:33.784548 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258117c4-1f0b-4fe7-8aad-5061640477f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:33.788278 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258117c4-1f0b-4fe7-8aad-5061640477f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:16:33.795966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-258117c4-1f0b-4fe7-8aad-5061640477f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:16:33.795966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:33.795966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:33.795966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:33.795966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:33.795966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:33.795966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:33.795966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:33.795966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:33.795966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:33.795966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:33.795966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:33.795966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:16:33.795966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:16:33.795966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:16:33.795966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:16:33.797125 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:16:33.797125 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:16:33.797125 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:16:33.797125 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:16:33.797125 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:33.797125 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:33.797125 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:33.797125 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:33.797125 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:33.797125 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:33.797125 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:33.797125 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:33.797125 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:33.797125 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:33.797125 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:33.797125 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:33.798366 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:33.798366 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:33.798366 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:33.798366 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:33.798366 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:16:33.798366 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:16:33.798366 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:16:33.798366 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:16:33.798366 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:16:33.798366 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:16:33.798366 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:16:33.798366 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:33.798366 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 540/1053] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Jul 7 15:16:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:16:33.803068 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258117c4-1f0b-4fe7-8aad-5061640477f2 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:33.812124 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 514/1054] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Jul 7 15:16:33 2022] POST /identity/v3/endpoints => generated 327 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:16:33.867033 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:33.917236 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258117c4-1f0b-4fe7-8aad-5061640477f2 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:33.967884 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb9e56-1be0-45f0-9cb6-51f3e1f75730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:33.968691 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb9e56-1be0-45f0-9cb6-51f3e1f75730 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:33.968836 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb9e56-1be0-45f0-9cb6-51f3e1f75730 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:33.969110 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb9e56-1be0-45f0-9cb6-51f3e1f75730 None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:33.969864 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfb9e56-1be0-45f0-9cb6-51f3e1f75730 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:33.970172 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfb9e56-1be0-45f0-9cb6-51f3e1f75730 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:33.971539 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfb9e56-1be0-45f0-9cb6-51f3e1f75730 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:33.981404 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 541/1055] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:16:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:33.987182 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1bfab-d8f3-4e79-8b1d-9eec5ca83b95 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:34.090827 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1bfab-d8f3-4e79-8b1d-9eec5ca83b95 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:34.104810 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:34.125514 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc8358-08e3-4bc2-a538-cc3d8d7188b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:34.126465 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc8358-08e3-4bc2-a538-cc3d8d7188b4 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:34.126750 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc8358-08e3-4bc2-a538-cc3d8d7188b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:34.127021 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc8358-08e3-4bc2-a538-cc3d8d7188b4 None admin] PATH_INFO: `/v3/users/815734a8b5a84efb9535b78cd1529dad` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:34.141212 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc8358-08e3-4bc2-a538-cc3d8d7188b4 None admin] RBAC: Authorizing `identity:get_user(user_id=815734a8b5a84efb9535b78cd1529dad)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:34.142639 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc8358-08e3-4bc2-a538-cc3d8d7188b4 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:34.144553 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 515/1056] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:16:33 2022] GET /identity/v3/users/815734a8b5a84efb9535b78cd1529dad => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:34.150262 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb9e56-1be0-45f0-9cb6-51f3e1f75730 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:34.255643 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb9e56-1be0-45f0-9cb6-51f3e1f75730 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:34.269369 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:34.289827 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493148a0-8e41-474d-b786-38d61e1afa5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:34.290557 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493148a0-8e41-474d-b786-38d61e1afa5f None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:34.290837 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493148a0-8e41-474d-b786-38d61e1afa5f None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:34.291105 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493148a0-8e41-474d-b786-38d61e1afa5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:34.358455 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493148a0-8e41-474d-b786-38d61e1afa5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:34.359841 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493148a0-8e41-474d-b786-38d61e1afa5f None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:34.459645 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 542/1057] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:16:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 07 15:16:34.461218 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc8358-08e3-4bc2-a538-cc3d8d7188b4 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:34.561903 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc8358-08e3-4bc2-a538-cc3d8d7188b4 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:34.598573 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88feb16e-d09d-442f-be44-7cb97bb2b87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:34.599791 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88feb16e-d09d-442f-be44-7cb97bb2b87b None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:34.600250 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88feb16e-d09d-442f-be44-7cb97bb2b87b None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:34.600714 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88feb16e-d09d-442f-be44-7cb97bb2b87b None admin] PATH_INFO: `/v3/projects/22fae5dcc60a4216bc1a506128557b5f` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:34.602063 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88feb16e-d09d-442f-be44-7cb97bb2b87b None admin] RBAC: Authorizing `identity:get_project(project_id=22fae5dcc60a4216bc1a506128557b5f)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:34.603689 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88feb16e-d09d-442f-be44-7cb97bb2b87b None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:34.605857 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 516/1058] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:16:34 2022] GET /identity/v3/projects/22fae5dcc60a4216bc1a506128557b5f => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:34.612136 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493148a0-8e41-474d-b786-38d61e1afa5f None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:34.626119 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:34.708215 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493148a0-8e41-474d-b786-38d61e1afa5f None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:34.743388 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8849ed-0391-46a6-8b03-045fa48063f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:34.744071 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8849ed-0391-46a6-8b03-045fa48063f9 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:34.744351 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8849ed-0391-46a6-8b03-045fa48063f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:34.744625 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8849ed-0391-46a6-8b03-045fa48063f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:34.745444 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8849ed-0391-46a6-8b03-045fa48063f9 None admin] RBAC: Adding query filter params (role.id=0e7793fbf3064037984da651879e3a94, scope.project.id=22fae5dcc60a4216bc1a506128557b5f, user.id=815734a8b5a84efb9535b78cd1529dad) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:34.745750 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8849ed-0391-46a6-8b03-045fa48063f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:34.747184 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8849ed-0391-46a6-8b03-045fa48063f9 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:34.752837 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 543/1059] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:16:34 2022] GET /identity/v3/role_assignments?user.id=815734a8b5a84efb9535b78cd1529dad&scope.project.id=22fae5dcc60a4216bc1a506128557b5f&role.id=0e7793fbf3064037984da651879e3a94 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:35.496173 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba9fdd-d40f-4af1-b032-5441d72da999 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:35.497243 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba9fdd-d40f-4af1-b032-5441d72da999 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:35.497352 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba9fdd-d40f-4af1-b032-5441d72da999 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:35.498795 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 517/1060] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:16:35.503732 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af772b1e-d16f-4b3f-af5e-0207caeb2920 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:35.504112 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af772b1e-d16f-4b3f-af5e-0207caeb2920 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:35.504287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af772b1e-d16f-4b3f-af5e-0207caeb2920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:35.522463 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-af772b1e-d16f-4b3f-af5e-0207caeb2920 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:35.523662 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-af772b1e-d16f-4b3f-af5e-0207caeb2920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:35.591191 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 544/1061] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:16:35.613788 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba9fdd-d40f-4af1-b032-5441d72da999 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:35.656585 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:35.718918 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba9fdd-d40f-4af1-b032-5441d72da999 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:35.754276 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20386c-cd5c-4d2b-b81f-4c2b754b6a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:35.755000 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20386c-cd5c-4d2b-b81f-4c2b754b6a1c None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:35.755276 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20386c-cd5c-4d2b-b81f-4c2b754b6a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:35.755543 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20386c-cd5c-4d2b-b81f-4c2b754b6a1c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:35.765690 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b20386c-cd5c-4d2b-b81f-4c2b754b6a1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:16:35.773757 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-4b20386c-cd5c-4d2b-b81f-4c2b754b6a1c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 07 15:16:35.773757 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:35.773757 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:35.773757 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:35.773757 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:35.773757 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:35.773757 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:35.773757 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:35.773757 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:35.773757 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:35.773757 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:35.773757 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:35.773757 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:16:35.773757 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:16:35.773757 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:16:35.773757 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:16:35.774693 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:16:35.774693 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:16:35.774693 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:16:35.774693 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:16:35.774693 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:35.774693 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:35.774693 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:35.774693 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:35.774693 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:35.774693 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:35.774693 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:35.774693 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:35.774693 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:35.774693 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:35.774693 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:35.774693 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:35.776002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:35.776002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:35.776002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:35.776002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:35.776002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:16:35.776002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:16:35.776002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:16:35.776002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:16:35.776002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:16:35.776002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:16:35.776002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 07 15:16:35.776002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:35.779922 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 518/1062] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Jul 7 15:16:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:35.781064 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af772b1e-d16f-4b3f-af5e-0207caeb2920 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:35.789339 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:35.862741 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41394bc-8d37-4931-bef0-60691a1b201e None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:35.863063 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41394bc-8d37-4931-bef0-60691a1b201e None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:35.863346 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41394bc-8d37-4931-bef0-60691a1b201e None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:35.864766 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 519/1063] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:16:35.869556 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e07539-87f2-488b-85a7-a23b79974f5e None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:35.869839 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e07539-87f2-488b-85a7-a23b79974f5e None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:35.870117 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e07539-87f2-488b-85a7-a23b79974f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:35.887306 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-58e07539-87f2-488b-85a7-a23b79974f5e None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:35.888535 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-58e07539-87f2-488b-85a7-a23b79974f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:35.891323 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af772b1e-d16f-4b3f-af5e-0207caeb2920 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:35.926546 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72699ed9-3982-4cda-aa7d-1eb0df4b5577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:35.927203 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72699ed9-3982-4cda-aa7d-1eb0df4b5577 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:35.927480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72699ed9-3982-4cda-aa7d-1eb0df4b5577 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:35.927774 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72699ed9-3982-4cda-aa7d-1eb0df4b5577 None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:35.928509 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72699ed9-3982-4cda-aa7d-1eb0df4b5577 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:35.928810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72699ed9-3982-4cda-aa7d-1eb0df4b5577 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:35.930164 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72699ed9-3982-4cda-aa7d-1eb0df4b5577 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:35.938702 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 545/1064] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Jul 7 15:16:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:35.944115 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72699ed9-3982-4cda-aa7d-1eb0df4b5577 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:35.964846 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 520/1065] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 07 15:16:35.987972 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e07539-87f2-488b-85a7-a23b79974f5e None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:36.019420 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:36.021037 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:36.054852 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72699ed9-3982-4cda-aa7d-1eb0df4b5577 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:36.092439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf0e06-9108-4917-82a2-89756d0698d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:36.093232 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf0e06-9108-4917-82a2-89756d0698d3 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:36.093514 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf0e06-9108-4917-82a2-89756d0698d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:36.093614 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e07539-87f2-488b-85a7-a23b79974f5e None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:36.093790 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf0e06-9108-4917-82a2-89756d0698d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:36.106643 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf0e06-9108-4917-82a2-89756d0698d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:36.108549 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf0e06-9108-4917-82a2-89756d0698d3 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:36.126392 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-33cf0e06-9108-4917-82a2-89756d0698d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:36.126392 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:36.126392 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:36.126392 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:36.126392 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:36.126392 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:36.126392 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:36.126392 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:36.126392 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:36.126392 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:36.126392 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:36.126392 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:36.126392 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:16:36.126392 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:16:36.126392 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:16:36.127308 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:16:36.127308 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:36.127308 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:36.127308 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:36.127308 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:36.127308 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:36.127308 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:36.127308 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:36.127308 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:36.127308 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:36.127308 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:36.127308 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:36.127308 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:36.127308 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:36.127308 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:36.127308 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:36.127308 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:36.128284 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:16:36.128284 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:16:36.128284 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:36.128284 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:36.132445 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 546/1066] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:16:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:16:36.133025 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf0e06-9108-4917-82a2-89756d0698d3 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:36.133143 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd031f-2f15-4bda-9996-8a9291404ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:36.133755 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd031f-2f15-4bda-9996-8a9291404ccf None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:36.133986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd031f-2f15-4bda-9996-8a9291404ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:36.134261 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd031f-2f15-4bda-9996-8a9291404ccf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:36.205949 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fd031f-2f15-4bda-9996-8a9291404ccf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:36.207381 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fd031f-2f15-4bda-9996-8a9291404ccf None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:36.238064 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf0e06-9108-4917-82a2-89756d0698d3 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:36.239920 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:36.259356 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:36.276234 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b202e4a-7648-448e-8910-5258e4afde8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:36.276836 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b202e4a-7648-448e-8910-5258e4afde8e None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:36.277174 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b202e4a-7648-448e-8910-5258e4afde8e None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:36.277548 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b202e4a-7648-448e-8910-5258e4afde8e None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:36.278845 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b202e4a-7648-448e-8910-5258e4afde8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:36.279152 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b202e4a-7648-448e-8910-5258e4afde8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:36.280563 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b202e4a-7648-448e-8910-5258e4afde8e None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:36.292533 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 547/1067] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:16:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:36.298518 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b202e4a-7648-448e-8910-5258e4afde8e None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:36.317436 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 521/1068] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:16:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 07 15:16:36.322918 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd031f-2f15-4bda-9996-8a9291404ccf None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:36.406788 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b202e4a-7648-448e-8910-5258e4afde8e None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:36.429195 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd031f-2f15-4bda-9996-8a9291404ccf None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:36.441937 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13183325-16a8-430b-aa01-7d25ca202b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:36.442730 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13183325-16a8-430b-aa01-7d25ca202b2a None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:36.443011 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13183325-16a8-430b-aa01-7d25ca202b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:36.443292 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13183325-16a8-430b-aa01-7d25ca202b2a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:36.451057 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13183325-16a8-430b-aa01-7d25ca202b2a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:36.452524 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13183325-16a8-430b-aa01-7d25ca202b2a None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:36.459783 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:36.466214 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2fe33f-991f-49b8-a4d8-6b6f7e610a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:36.466999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2fe33f-991f-49b8-a4d8-6b6f7e610a79 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:36.467251 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-13183325-16a8-430b-aa01-7d25ca202b2a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 07 15:16:36.467251 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:36.467251 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:36.467251 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:36.467251 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:36.467251 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:36.467251 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:36.467251 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:36.467251 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:36.467251 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:36.467251 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:36.467251 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:36.467251 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 15:16:36.467251 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 15:16:36.467251 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 15:16:36.467251 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 15:16:36.468188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:36.468188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:36.468188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 15:16:36.468188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:16:36.468188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 15:16:36.468188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:16:36.468188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:36.468188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:36.468188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:36.468188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:36.468188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:36.468188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:36.468188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:36.468188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:36.468188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:36.468188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:36.468188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:36.468188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:36.469188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:36.469188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:36.469188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 15:16:36.469188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 15:16:36.469188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 15:16:36.469188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 15:16:36.469188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 15:16:36.469188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 15:16:36.469188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 07 15:16:36.469188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:36.469735 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2fe33f-991f-49b8-a4d8-6b6f7e610a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:36.469735 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2fe33f-991f-49b8-a4d8-6b6f7e610a79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:36.469878 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 548/1069] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Jul 7 15:16:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:16:36.474260 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13183325-16a8-430b-aa01-7d25ca202b2a None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:36.480794 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2fe33f-991f-49b8-a4d8-6b6f7e610a79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:36.482253 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2fe33f-991f-49b8-a4d8-6b6f7e610a79 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:36.499480 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-9f2fe33f-991f-49b8-a4d8-6b6f7e610a79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:36.499480 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:36.499480 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:36.499480 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:36.499480 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:36.499480 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:36.499480 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:36.499480 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:36.499480 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:36.499480 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:36.499480 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:36.499480 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:36.499480 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:16:36.499480 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:16:36.499480 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:16:36.501054 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:16:36.501054 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:36.501054 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:36.501054 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:36.501054 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:36.501054 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:36.501054 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:36.501054 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:36.501054 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:36.501054 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:36.501054 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:36.501054 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:36.501054 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:36.501054 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:36.501054 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:36.501054 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:36.501054 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:36.501988 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:16:36.501988 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:16:36.501988 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:36.501988 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:36.501988 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 522/1070] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:16:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:16:36.506777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2fe33f-991f-49b8-a4d8-6b6f7e610a79 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:36.520710 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:36.579628 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13183325-16a8-430b-aa01-7d25ca202b2a None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:36.609719 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2fe33f-991f-49b8-a4d8-6b6f7e610a79 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:36.619933 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749da023-9a2f-4cc4-ab19-e1fcb5724fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:36.620915 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749da023-9a2f-4cc4-ab19-e1fcb5724fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:36.621490 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749da023-9a2f-4cc4-ab19-e1fcb5724fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:36.621775 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749da023-9a2f-4cc4-ab19-e1fcb5724fc8 None admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:36.622772 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749da023-9a2f-4cc4-ab19-e1fcb5724fc8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:36.623087 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749da023-9a2f-4cc4-ab19-e1fcb5724fc8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:36.625208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749da023-9a2f-4cc4-ab19-e1fcb5724fc8 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:36.643505 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac3ce9-d369-4002-9177-562ce05d42c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:36.644190 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac3ce9-d369-4002-9177-562ce05d42c0 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:36.644482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac3ce9-d369-4002-9177-562ce05d42c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:36.644729 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac3ce9-d369-4002-9177-562ce05d42c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:36.645446 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ac3ce9-d369-4002-9177-562ce05d42c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:36.645748 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ac3ce9-d369-4002-9177-562ce05d42c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:36.651025 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 549/1071] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Jul 7 15:16:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:36.651180 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ac3ce9-d369-4002-9177-562ce05d42c0 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:36.652036 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749da023-9a2f-4cc4-ab19-e1fcb5724fc8 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:36.660108 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 523/1072] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:16:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:36.744660 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94d7ea-68ec-4ea4-9645-a552ccaa88a6 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:36.744948 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94d7ea-68ec-4ea4-9645-a552ccaa88a6 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:36.745231 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94d7ea-68ec-4ea4-9645-a552ccaa88a6 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:36.747189 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 524/1073] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:16:36.752992 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f3c62-d6cc-4e7c-85d0-12af1cefc4ba None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:36.753363 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f3c62-d6cc-4e7c-85d0-12af1cefc4ba None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:36.753710 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f3c62-d6cc-4e7c-85d0-12af1cefc4ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:36.770458 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:36.772322 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-be7f3c62-d6cc-4e7c-85d0-12af1cefc4ba None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:36.773507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-be7f3c62-d6cc-4e7c-85d0-12af1cefc4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:36.774750 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749da023-9a2f-4cc4-ab19-e1fcb5724fc8 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:36.814255 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64dfa61-0d9d-4e16-8861-85310fc6d3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:36.814660 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64dfa61-0d9d-4e16-8861-85310fc6d3bd None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:36.814937 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64dfa61-0d9d-4e16-8861-85310fc6d3bd None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:36.815207 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64dfa61-0d9d-4e16-8861-85310fc6d3bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:36.817858 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:36.829373 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64dfa61-0d9d-4e16-8861-85310fc6d3bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:36.833519 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64dfa61-0d9d-4e16-8861-85310fc6d3bd None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:36.853645 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-d64dfa61-0d9d-4e16-8861-85310fc6d3bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:36.853645 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:36.853645 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:36.853645 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:36.853645 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:36.853645 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:36.853645 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:36.853645 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:36.853645 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:36.853645 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:36.853645 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:36.853645 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:36.853645 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:16:36.853645 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:16:36.853645 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:16:36.854785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:16:36.854785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:36.854785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:36.854785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:36.854785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:36.854785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:36.854785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:36.854785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:36.854785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:36.854785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:36.854785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:36.854785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:36.854785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:36.854785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:36.854785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:36.854785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:36.854785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:36.855804 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:16:36.855804 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:16:36.855804 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:36.855804 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:36.855804 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 550/1075] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:16:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:36.859781 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 525/1075] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 07 15:16:36.860690 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64dfa61-0d9d-4e16-8861-85310fc6d3bd None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:36.874159 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f3c62-d6cc-4e7c-85d0-12af1cefc4ba None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:36.979539 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64dfa61-0d9d-4e16-8861-85310fc6d3bd None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:36.992307 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f3c62-d6cc-4e7c-85d0-12af1cefc4ba None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:37.013848 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b77572-604e-4e90-b250-9ed674ff9a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:37.014506 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b77572-604e-4e90-b250-9ed674ff9a93 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:37.014791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b77572-604e-4e90-b250-9ed674ff9a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:37.015071 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b77572-604e-4e90-b250-9ed674ff9a93 None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:37.015825 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b77572-604e-4e90-b250-9ed674ff9a93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:37.016137 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b77572-604e-4e90-b250-9ed674ff9a93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:37.017509 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b77572-604e-4e90-b250-9ed674ff9a93 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:37.026487 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1eef6b-861c-44c2-a65f-f1b79ba6dca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:37.027266 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1eef6b-861c-44c2-a65f-f1b79ba6dca0 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:37.027548 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1eef6b-861c-44c2-a65f-f1b79ba6dca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:37.027879 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1eef6b-861c-44c2-a65f-f1b79ba6dca0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:37.028567 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 551/1076] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:16:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:37.033943 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b77572-604e-4e90-b250-9ed674ff9a93 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:37.038370 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1eef6b-861c-44c2-a65f-f1b79ba6dca0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:16:37.045793 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:37.046068 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-bf1eef6b-861c-44c2-a65f-f1b79ba6dca0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:16:37.046068 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:37.046068 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:37.046068 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:37.046068 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:37.046068 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:37.046068 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:37.046068 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:37.046068 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:37.046068 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:37.046068 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:37.046068 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:37.046068 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:16:37.046068 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:16:37.046068 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:16:37.046068 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:16:37.047227 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:16:37.047227 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:16:37.047227 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:16:37.047227 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:16:37.047227 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:37.047227 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:37.047227 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:37.047227 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:37.047227 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:37.047227 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:37.047227 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:37.047227 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:37.047227 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:37.047227 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:37.047227 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:37.047227 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:37.048139 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:37.048139 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:37.048139 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:37.048139 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:37.048139 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:16:37.048139 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:16:37.048139 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:16:37.048139 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:16:37.048139 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:16:37.048139 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:16:37.048139 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:16:37.048139 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:37.048139 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 526/1077] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Jul 7 15:16:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:16:37.053279 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1eef6b-861c-44c2-a65f-f1b79ba6dca0 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:37.085742 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:37.134284 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b77572-604e-4e90-b250-9ed674ff9a93 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:37.153585 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1eef6b-861c-44c2-a65f-f1b79ba6dca0 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:37.166940 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac21517-2264-44be-995a-fd9872e1c5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:37.167784 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac21517-2264-44be-995a-fd9872e1c5cd None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:37.168071 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac21517-2264-44be-995a-fd9872e1c5cd None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:37.168878 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac21517-2264-44be-995a-fd9872e1c5cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:37.175399 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac21517-2264-44be-995a-fd9872e1c5cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:37.176891 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac21517-2264-44be-995a-fd9872e1c5cd None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:37.187772 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da9278-802e-4fdd-8570-7e07c7b60300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:37.188424 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da9278-802e-4fdd-8570-7e07c7b60300 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:37.188710 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da9278-802e-4fdd-8570-7e07c7b60300 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:37.188990 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da9278-802e-4fdd-8570-7e07c7b60300 None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:37.189732 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da9278-802e-4fdd-8570-7e07c7b60300 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:37.190040 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da9278-802e-4fdd-8570-7e07c7b60300 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:37.191391 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da9278-802e-4fdd-8570-7e07c7b60300 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:37.191482 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-eac21517-2264-44be-995a-fd9872e1c5cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:16:37.191482 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:37.191482 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:37.191482 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:37.191482 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:37.191482 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:37.191482 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:37.191482 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:37.191482 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:37.191482 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:37.191482 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:37.191482 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:37.191482 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 15:16:37.191482 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 15:16:37.191482 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 15:16:37.192485 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 15:16:37.192485 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:37.192485 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:37.192485 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:37.192485 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:37.192485 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:37.192485 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:37.192485 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:37.192485 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:37.192485 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:37.192485 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:37.192485 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:37.192485 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:37.192485 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:37.192485 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:37.192485 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:37.192485 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:37.193413 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 15:16:37.193413 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 15:16:37.193413 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 15:16:37.193413 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 15:16:37.193413 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 15:16:37.193413 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 15:16:37.193413 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:16:37.193413 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:37.193413 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 552/1078] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:16:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:16:37.198374 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac21517-2264-44be-995a-fd9872e1c5cd None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:37.206135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 527/1079] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:16:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:37.207150 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da9278-802e-4fdd-8570-7e07c7b60300 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:37.266953 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:37.297954 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac21517-2264-44be-995a-fd9872e1c5cd None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:37.300535 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:37.307684 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da9278-802e-4fdd-8570-7e07c7b60300 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:37.331837 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c81ca0-8697-47b3-9526-a07557f50dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:37.332495 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c81ca0-8697-47b3-9526-a07557f50dca None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:37.332782 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c81ca0-8697-47b3-9526-a07557f50dca None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:37.333059 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c81ca0-8697-47b3-9526-a07557f50dca None admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:37.333842 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c81ca0-8697-47b3-9526-a07557f50dca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:37.334152 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c81ca0-8697-47b3-9526-a07557f50dca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:37.335594 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c81ca0-8697-47b3-9526-a07557f50dca None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:37.342250 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372feee9-8c1e-4bfb-a9ef-9af2c429f180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:37.343050 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372feee9-8c1e-4bfb-a9ef-9af2c429f180 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:37.343334 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372feee9-8c1e-4bfb-a9ef-9af2c429f180 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:37.343641 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372feee9-8c1e-4bfb-a9ef-9af2c429f180 None admin] PATH_INFO: `/v3/users/815734a8b5a84efb9535b78cd1529dad` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:37.344844 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372feee9-8c1e-4bfb-a9ef-9af2c429f180 None admin] RBAC: Authorizing `identity:get_user(user_id=815734a8b5a84efb9535b78cd1529dad)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:37.346263 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372feee9-8c1e-4bfb-a9ef-9af2c429f180 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:37.347083 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 553/1080] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Jul 7 15:16:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:37.351592 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 528/1081] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:16:37 2022] GET /identity/v3/users/815734a8b5a84efb9535b78cd1529dad => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:37.353044 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c81ca0-8697-47b3-9526-a07557f50dca None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:37.353591 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372feee9-8c1e-4bfb-a9ef-9af2c429f180 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:37.452858 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372feee9-8c1e-4bfb-a9ef-9af2c429f180 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:37.453070 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c81ca0-8697-47b3-9526-a07557f50dca None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:37.480505 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:37.486806 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226e1ff7-f83e-48e8-8238-b99e11733fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:37.487363 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe0d63-bd24-460c-9362-6f0d1b81c278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:37.487715 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e1ff7-f83e-48e8-8238-b99e11733fb6 None admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:37.487997 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e1ff7-f83e-48e8-8238-b99e11733fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:37.488256 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe0d63-bd24-460c-9362-6f0d1b81c278 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:37.488351 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e1ff7-f83e-48e8-8238-b99e11733fb6 None admin] PATH_INFO: `/v3/projects/c5260c7fd3614e9e93b167c86a94260e/users/de1632555cb345f2b9213da1925c02f4/roles/d5899b9e6b9940aa9837652e51a23908` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:37.488658 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe0d63-bd24-460c-9362-6f0d1b81c278 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:37.488941 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe0d63-bd24-460c-9362-6f0d1b81c278 None admin] PATH_INFO: `/v3/projects/22fae5dcc60a4216bc1a506128557b5f` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:37.490112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe0d63-bd24-460c-9362-6f0d1b81c278 None admin] RBAC: Authorizing `identity:get_project(project_id=22fae5dcc60a4216bc1a506128557b5f)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:37.490468 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226e1ff7-f83e-48e8-8238-b99e11733fb6 None admin] RBAC: Authorizing `identity:create_grant(project_id=c5260c7fd3614e9e93b167c86a94260e, user_id=de1632555cb345f2b9213da1925c02f4, role_id=d5899b9e6b9940aa9837652e51a23908)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:37.491536 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe0d63-bd24-460c-9362-6f0d1b81c278 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:37.491968 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226e1ff7-f83e-48e8-8238-b99e11733fb6 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:37.493261 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 529/1082] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:16:37 2022] GET /identity/v3/projects/22fae5dcc60a4216bc1a506128557b5f => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:37.499659 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe0d63-bd24-460c-9362-6f0d1b81c278 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:37.528988 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:37.578998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 554/1083] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Jul 7 15:16:37 2022] PUT /identity/v3/projects/c5260c7fd3614e9e93b167c86a94260e/users/de1632555cb345f2b9213da1925c02f4/roles/d5899b9e6b9940aa9837652e51a23908 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:16:37.597035 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe0d63-bd24-460c-9362-6f0d1b81c278 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:37.635479 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e7ed6-b1f1-46c0-b509-2812624bffec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:37.636414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e7ed6-b1f1-46c0-b509-2812624bffec None admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:37.636695 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e7ed6-b1f1-46c0-b509-2812624bffec None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:37.636980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e7ed6-b1f1-46c0-b509-2812624bffec None admin] PATH_INFO: `/v3/projects/22fae5dcc60a4216bc1a506128557b5f/users/815734a8b5a84efb9535b78cd1529dad/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:37.639009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2e7ed6-b1f1-46c0-b509-2812624bffec None admin] RBAC: Authorizing `identity:create_grant(project_id=22fae5dcc60a4216bc1a506128557b5f, user_id=815734a8b5a84efb9535b78cd1529dad, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:37.640487 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2e7ed6-b1f1-46c0-b509-2812624bffec None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:37.711427 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 530/1084] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Jul 7 15:16:37 2022] PUT /identity/v3/projects/22fae5dcc60a4216bc1a506128557b5f/users/815734a8b5a84efb9535b78cd1529dad/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:16:39.607730 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dbd3a8-fab6-4316-8c22-574adf9a93a0 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:39.608257 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dbd3a8-fab6-4316-8c22-574adf9a93a0 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:39.608257 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dbd3a8-fab6-4316-8c22-574adf9a93a0 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:39.609913 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 555/1085] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:16:39.614904 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ecd956-b224-4c9b-84bb-33d8575b3fac None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:39.615277 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ecd956-b224-4c9b-84bb-33d8575b3fac None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:39.615468 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ecd956-b224-4c9b-84bb-33d8575b3fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:39.631955 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-49ecd956-b224-4c9b-84bb-33d8575b3fac None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:39.633136 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-49ecd956-b224-4c9b-84bb-33d8575b3fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:39.694061 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f2d59c-bd2d-4dfb-87a7-5266d285b341 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:39.694347 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f2d59c-bd2d-4dfb-87a7-5266d285b341 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:39.694626 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f2d59c-bd2d-4dfb-87a7-5266d285b341 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:39.696006 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 556/1086] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:16:39.699072 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 531/1087] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:16:39.701417 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708e873-ac31-4d94-9bd5-eaa4fcf3ea32 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:39.701705 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708e873-ac31-4d94-9bd5-eaa4fcf3ea32 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:39.701985 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708e873-ac31-4d94-9bd5-eaa4fcf3ea32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:39.723313 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-0708e873-ac31-4d94-9bd5-eaa4fcf3ea32 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:39.724587 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-0708e873-ac31-4d94-9bd5-eaa4fcf3ea32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:39.729734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ecd956-b224-4c9b-84bb-33d8575b3fac None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:39.751043 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:39.792929 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 557/1088] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:16:39.816414 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708e873-ac31-4d94-9bd5-eaa4fcf3ea32 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:39.829637 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ecd956-b224-4c9b-84bb-33d8575b3fac None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:39.863156 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c352fa17-b0d7-4793-8666-083850e66f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:39.863915 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352fa17-b0d7-4793-8666-083850e66f1f None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:39.864197 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352fa17-b0d7-4793-8666-083850e66f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:39.864470 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352fa17-b0d7-4793-8666-083850e66f1f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:39.874926 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c352fa17-b0d7-4793-8666-083850e66f1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:16:39.883104 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-c352fa17-b0d7-4793-8666-083850e66f1f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 07 15:16:39.883104 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:39.883104 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:39.883104 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:39.883104 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:39.883104 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:39.883104 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:39.883104 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:39.883104 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:39.883104 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:39.883104 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:39.883104 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:39.883104 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:16:39.883104 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:16:39.883104 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:16:39.883104 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:16:39.885979 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:16:39.885979 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:16:39.885979 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:16:39.885979 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:16:39.885979 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:39.885979 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:39.885979 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:39.885979 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:39.885979 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:39.885979 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:39.885979 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:39.885979 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:39.885979 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:39.885979 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:39.885979 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:39.885979 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:39.886852 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:39.886852 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:39.886852 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:39.886852 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:39.886852 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:16:39.886852 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:16:39.886852 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:16:39.886852 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:16:39.886852 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:16:39.886852 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:16:39.886852 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 07 15:16:39.886852 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:39.886852 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 532/1089] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Jul 7 15:16:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:16:39.891067 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c352fa17-b0d7-4793-8666-083850e66f1f None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:39.914041 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708e873-ac31-4d94-9bd5-eaa4fcf3ea32 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:39.933384 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:39.947723 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f39146-d0e4-4d66-90a9-f24e87e51194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:39.948537 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f39146-d0e4-4d66-90a9-f24e87e51194 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:39.948840 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f39146-d0e4-4d66-90a9-f24e87e51194 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:39.949194 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f39146-d0e4-4d66-90a9-f24e87e51194 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:39.961081 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f39146-d0e4-4d66-90a9-f24e87e51194 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:16:39.968353 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-47f39146-d0e4-4d66-90a9-f24e87e51194 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:16:39.968353 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:39.968353 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:39.968353 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:39.968353 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:39.968353 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:39.968353 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:39.968353 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:39.968353 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:39.968353 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:39.968353 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:39.968353 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:39.968353 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:16:39.968353 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:16:39.968353 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:16:39.968353 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:16:39.969381 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:16:39.969381 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:16:39.969381 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:16:39.969381 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:16:39.969381 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:39.969381 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:39.969381 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:39.969381 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:39.969381 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:39.969381 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:39.969381 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:39.969381 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:39.969381 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:39.969381 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:39.969381 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:39.969381 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:39.970322 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:39.970322 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:39.970322 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:39.970322 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:39.970322 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:16:39.970322 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:16:39.970322 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:16:39.970322 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:16:39.970322 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:16:39.970322 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:16:39.970322 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:16:39.970322 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:39.970322 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 558/1090] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Jul 7 15:16:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:16:39.976432 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f39146-d0e4-4d66-90a9-f24e87e51194 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:39.980975 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:39.995469 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c352fa17-b0d7-4793-8666-083850e66f1f None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:40.030443 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba00113-bbe1-4c25-a6a1-b02657ddeaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:40.031099 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba00113-bbe1-4c25-a6a1-b02657ddeaa5 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:40.031375 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba00113-bbe1-4c25-a6a1-b02657ddeaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:40.031675 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba00113-bbe1-4c25-a6a1-b02657ddeaa5 None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:40.032412 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba00113-bbe1-4c25-a6a1-b02657ddeaa5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:40.032777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba00113-bbe1-4c25-a6a1-b02657ddeaa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:40.034181 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba00113-bbe1-4c25-a6a1-b02657ddeaa5 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:40.042352 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 533/1091] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Jul 7 15:16:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:40.048215 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba00113-bbe1-4c25-a6a1-b02657ddeaa5 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:40.076428 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f39146-d0e4-4d66-90a9-f24e87e51194 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:40.110673 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b0f34-533b-4af6-9780-2ff5bef0c8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:40.111364 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b0f34-533b-4af6-9780-2ff5bef0c8b3 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:40.111670 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b0f34-533b-4af6-9780-2ff5bef0c8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:40.111953 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b0f34-533b-4af6-9780-2ff5bef0c8b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:40.112694 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921b0f34-533b-4af6-9780-2ff5bef0c8b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:40.113006 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921b0f34-533b-4af6-9780-2ff5bef0c8b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:40.114409 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921b0f34-533b-4af6-9780-2ff5bef0c8b3 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:40.122953 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 559/1092] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:16:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:40.128703 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b0f34-533b-4af6-9780-2ff5bef0c8b3 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:40.150998 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba00113-bbe1-4c25-a6a1-b02657ddeaa5 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:40.170810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:40.184768 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caab1b9-1d2a-4813-9c9e-f2bdc4f6117e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:40.185550 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caab1b9-1d2a-4813-9c9e-f2bdc4f6117e None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:40.185833 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caab1b9-1d2a-4813-9c9e-f2bdc4f6117e None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:40.186112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caab1b9-1d2a-4813-9c9e-f2bdc4f6117e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:40.196491 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caab1b9-1d2a-4813-9c9e-f2bdc4f6117e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:40.197955 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caab1b9-1d2a-4813-9c9e-f2bdc4f6117e None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:40.199731 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:40.215280 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-5caab1b9-1d2a-4813-9c9e-f2bdc4f6117e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:40.215280 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:40.215280 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:40.215280 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:40.215280 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:40.215280 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:40.215280 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:40.215280 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:40.215280 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:40.215280 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:40.215280 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:40.215280 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:40.215280 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:16:40.215280 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:16:40.215280 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:16:40.216251 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:16:40.216251 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:40.216251 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:40.216251 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:40.216251 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:40.216251 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:40.216251 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:40.216251 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:40.216251 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:40.216251 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:40.216251 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:40.216251 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:40.216251 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:40.216251 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:40.216251 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:40.216251 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:40.216251 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:40.217595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:16:40.217595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:16:40.217595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:40.217595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:40.217595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 534/1093] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:16:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:16:40.222077 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caab1b9-1d2a-4813-9c9e-f2bdc4f6117e None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:40.228146 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b0f34-533b-4af6-9780-2ff5bef0c8b3 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:40.262193 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22fda8d-e4a6-4733-9065-8e6d23e956f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:40.263058 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22fda8d-e4a6-4733-9065-8e6d23e956f9 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:40.263336 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22fda8d-e4a6-4733-9065-8e6d23e956f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:40.263632 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22fda8d-e4a6-4733-9065-8e6d23e956f9 None admin] PATH_INFO: `/v3/users/815734a8b5a84efb9535b78cd1529dad` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:40.264818 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22fda8d-e4a6-4733-9065-8e6d23e956f9 None admin] RBAC: Authorizing `identity:get_user(user_id=815734a8b5a84efb9535b78cd1529dad)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:40.266235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22fda8d-e4a6-4733-9065-8e6d23e956f9 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:40.267879 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 560/1094] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:16:40 2022] GET /identity/v3/users/815734a8b5a84efb9535b78cd1529dad => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:40.273539 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22fda8d-e4a6-4733-9065-8e6d23e956f9 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:40.323806 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caab1b9-1d2a-4813-9c9e-f2bdc4f6117e None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:40.357271 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdbd98e-be38-491f-9b32-397b2692c17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:40.357926 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdbd98e-be38-491f-9b32-397b2692c17e None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:40.358209 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdbd98e-be38-491f-9b32-397b2692c17e None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:40.358486 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdbd98e-be38-491f-9b32-397b2692c17e None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:40.359202 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdbd98e-be38-491f-9b32-397b2692c17e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:40.359508 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdbd98e-be38-491f-9b32-397b2692c17e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:40.360885 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdbd98e-be38-491f-9b32-397b2692c17e None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:40.370550 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22fda8d-e4a6-4733-9065-8e6d23e956f9 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:40.372860 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 535/1095] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:16:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:40.378814 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdbd98e-be38-491f-9b32-397b2692c17e None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:40.381327 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:40.404603 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d0f437-6942-4deb-ac08-40cdc959f34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:40.405246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0f437-6942-4deb-ac08-40cdc959f34b None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:40.405526 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0f437-6942-4deb-ac08-40cdc959f34b None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:40.405818 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0f437-6942-4deb-ac08-40cdc959f34b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:40.432981 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:40.475011 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d0f437-6942-4deb-ac08-40cdc959f34b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:40.476575 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d0f437-6942-4deb-ac08-40cdc959f34b None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:40.477535 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdbd98e-be38-491f-9b32-397b2692c17e None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:40.510789 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70234dbe-bb49-42c2-a66d-b25a500bffc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:40.511557 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70234dbe-bb49-42c2-a66d-b25a500bffc4 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:40.511887 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70234dbe-bb49-42c2-a66d-b25a500bffc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:40.512160 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70234dbe-bb49-42c2-a66d-b25a500bffc4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:40.519875 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70234dbe-bb49-42c2-a66d-b25a500bffc4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:40.521325 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70234dbe-bb49-42c2-a66d-b25a500bffc4 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:40.535550 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-70234dbe-bb49-42c2-a66d-b25a500bffc4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 07 15:16:40.535550 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:40.535550 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:40.535550 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:40.535550 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:40.535550 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:40.535550 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:40.535550 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:40.535550 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:40.535550 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:40.535550 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:40.535550 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:40.535550 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 15:16:40.535550 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 15:16:40.535550 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 15:16:40.535550 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 15:16:40.536543 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:40.536543 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:40.536543 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 15:16:40.536543 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:16:40.536543 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 15:16:40.536543 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:16:40.536543 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:40.536543 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:40.536543 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:40.536543 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:40.536543 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:40.536543 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:40.536543 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:40.536543 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:40.536543 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:40.536543 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:40.536543 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:40.536543 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:40.537520 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:40.537520 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:40.537520 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 15:16:40.537520 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 15:16:40.537520 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 15:16:40.537520 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 15:16:40.537520 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 15:16:40.537520 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 15:16:40.537520 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 07 15:16:40.537520 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:40.537520 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 536/1096] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Jul 7 15:16:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:16:40.542582 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70234dbe-bb49-42c2-a66d-b25a500bffc4 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:40.575959 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 561/1097] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:16:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 07 15:16:40.579246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d0f437-6942-4deb-ac08-40cdc959f34b None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:40.590546 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:40.642132 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70234dbe-bb49-42c2-a66d-b25a500bffc4 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:40.655595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:40.675296 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18855c26-5540-44b4-b1d8-9106a6a91e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:40.675976 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18855c26-5540-44b4-b1d8-9106a6a91e79 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:40.676262 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d0f437-6942-4deb-ac08-40cdc959f34b None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:40.676689 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18855c26-5540-44b4-b1d8-9106a6a91e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:40.676689 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18855c26-5540-44b4-b1d8-9106a6a91e79 None admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:40.677389 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18855c26-5540-44b4-b1d8-9106a6a91e79 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:40.677701 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18855c26-5540-44b4-b1d8-9106a6a91e79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:40.679151 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18855c26-5540-44b4-b1d8-9106a6a91e79 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:40.696195 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 537/1098] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Jul 7 15:16:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:40.701899 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18855c26-5540-44b4-b1d8-9106a6a91e79 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:40.714870 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5295b085-fc3d-4e73-8f98-73aad056f0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:40.716272 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295b085-fc3d-4e73-8f98-73aad056f0ce None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:40.716547 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295b085-fc3d-4e73-8f98-73aad056f0ce None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:40.716815 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295b085-fc3d-4e73-8f98-73aad056f0ce None admin] PATH_INFO: `/v3/projects/22fae5dcc60a4216bc1a506128557b5f` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:40.717993 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5295b085-fc3d-4e73-8f98-73aad056f0ce None admin] RBAC: Authorizing `identity:get_project(project_id=22fae5dcc60a4216bc1a506128557b5f)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:40.719399 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5295b085-fc3d-4e73-8f98-73aad056f0ce None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:40.721150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 562/1099] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:16:40 2022] GET /identity/v3/projects/22fae5dcc60a4216bc1a506128557b5f => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:40.727312 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5295b085-fc3d-4e73-8f98-73aad056f0ce None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:40.800819 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18855c26-5540-44b4-b1d8-9106a6a91e79 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:40.804847 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:40.825289 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5295b085-fc3d-4e73-8f98-73aad056f0ce None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:40.835216 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f089259e-159f-4f67-9298-52aeac2931e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:40.835889 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089259e-159f-4f67-9298-52aeac2931e1 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:40.836175 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089259e-159f-4f67-9298-52aeac2931e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:40.836445 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089259e-159f-4f67-9298-52aeac2931e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:40.859997 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb4c0f3-eb35-44a1-9596-a319fbd8e87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:40.860635 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb4c0f3-eb35-44a1-9596-a319fbd8e87e None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:40.860915 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb4c0f3-eb35-44a1-9596-a319fbd8e87e None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:40.861192 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb4c0f3-eb35-44a1-9596-a319fbd8e87e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:40.862006 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb4c0f3-eb35-44a1-9596-a319fbd8e87e None admin] RBAC: Adding query filter params (role.id=0e7793fbf3064037984da651879e3a94, scope.project.id=22fae5dcc60a4216bc1a506128557b5f, user.id=815734a8b5a84efb9535b78cd1529dad) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:40.862314 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb4c0f3-eb35-44a1-9596-a319fbd8e87e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:40.863747 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb4c0f3-eb35-44a1-9596-a319fbd8e87e None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:40.869371 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 563/1100] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:16:40 2022] GET /identity/v3/role_assignments?user.id=815734a8b5a84efb9535b78cd1529dad&scope.project.id=22fae5dcc60a4216bc1a506128557b5f&role.id=0e7793fbf3064037984da651879e3a94 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:40.878673 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:40.907365 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f089259e-159f-4f67-9298-52aeac2931e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:40.908772 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f089259e-159f-4f67-9298-52aeac2931e1 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:41.019124 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 538/1101] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:16:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 07 15:16:41.025055 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb4c0f3-eb35-44a1-9596-a319fbd8e87e None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:41.124079 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb4c0f3-eb35-44a1-9596-a319fbd8e87e None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:41.157719 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5860f12-cc93-4243-ba8b-8f414f4d3eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:41.158423 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5860f12-cc93-4243-ba8b-8f414f4d3eda None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:41.158700 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5860f12-cc93-4243-ba8b-8f414f4d3eda None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:41.158966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5860f12-cc93-4243-ba8b-8f414f4d3eda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:41.169009 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5860f12-cc93-4243-ba8b-8f414f4d3eda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:41.170443 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5860f12-cc93-4243-ba8b-8f414f4d3eda None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:41.172187 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:41.188072 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-e5860f12-cc93-4243-ba8b-8f414f4d3eda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:41.188072 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:41.188072 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:41.188072 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:41.188072 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:41.188072 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:41.188072 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:41.188072 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:41.188072 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:41.188072 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:41.188072 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:41.188072 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:41.188072 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:16:41.188072 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:16:41.188072 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:16:41.189217 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:16:41.189217 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:41.189217 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:41.189217 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:41.189217 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:41.189217 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:41.189217 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:41.189217 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:41.189217 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:41.189217 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:41.189217 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:41.189217 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:41.189217 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:41.189217 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:41.189217 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:41.189217 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:41.189217 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:41.190311 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:16:41.190311 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:16:41.190311 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:16:41.190311 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:41.190734 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 564/1102] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:16:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 15:16:41.194880 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f089259e-159f-4f67-9298-52aeac2931e1 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:41.263511 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:41.291680 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f089259e-159f-4f67-9298-52aeac2931e1 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:41.325449 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fdad09-2c76-4101-828b-3747fe98493a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:41.326104 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fdad09-2c76-4101-828b-3747fe98493a None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:41.326432 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fdad09-2c76-4101-828b-3747fe98493a None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:41.326716 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fdad09-2c76-4101-828b-3747fe98493a None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:41.327429 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fdad09-2c76-4101-828b-3747fe98493a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:41.327759 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fdad09-2c76-4101-828b-3747fe98493a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:41.329104 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fdad09-2c76-4101-828b-3747fe98493a None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:41.340916 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 539/1103] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:16:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:41.346475 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5860f12-cc93-4243-ba8b-8f414f4d3eda None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:41.444134 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5860f12-cc93-4243-ba8b-8f414f4d3eda None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:41.477590 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a388eb-6c09-4a3a-8e53-6aaa2706a274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:41.478373 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a388eb-6c09-4a3a-8e53-6aaa2706a274 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:41.478652 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a388eb-6c09-4a3a-8e53-6aaa2706a274 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:41.478921 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a388eb-6c09-4a3a-8e53-6aaa2706a274 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:41.485462 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a388eb-6c09-4a3a-8e53-6aaa2706a274 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:41.486893 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a388eb-6c09-4a3a-8e53-6aaa2706a274 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:41.500759 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-97a388eb-6c09-4a3a-8e53-6aaa2706a274 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:16:41.500759 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:41.500759 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:41.500759 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:41.500759 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:41.500759 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:41.500759 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:41.500759 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:41.500759 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:41.500759 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:41.500759 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:41.500759 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:41.500759 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 15:16:41.500759 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 15:16:41.500759 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 15:16:41.501751 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 15:16:41.501751 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:41.501751 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:41.501751 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:41.501751 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:41.501751 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:41.501751 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:41.501751 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:41.501751 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:41.501751 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:41.501751 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:41.501751 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:41.501751 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:41.501751 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:41.501751 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:41.501751 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:41.501751 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:41.502665 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 15:16:41.502665 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 15:16:41.502665 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 15:16:41.502665 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 15:16:41.502665 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 15:16:41.502665 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 15:16:41.502665 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:16:41.502665 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:16:41.502665 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 565/1104] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:16:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:41.507786 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fdad09-2c76-4101-828b-3747fe98493a None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:41.606730 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fdad09-2c76-4101-828b-3747fe98493a None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:41.634851 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:41.640317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f3928-c52c-495d-ab3a-cd1d7cf02482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:41.640965 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f3928-c52c-495d-ab3a-cd1d7cf02482 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:41.641243 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f3928-c52c-495d-ab3a-cd1d7cf02482 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:41.641514 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f3928-c52c-495d-ab3a-cd1d7cf02482 None admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:41.642287 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579f3928-c52c-495d-ab3a-cd1d7cf02482 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:41.642807 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579f3928-c52c-495d-ab3a-cd1d7cf02482 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:41.644021 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579f3928-c52c-495d-ab3a-cd1d7cf02482 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:41.655871 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 540/1105] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Jul 7 15:16:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:41.661751 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a388eb-6c09-4a3a-8e53-6aaa2706a274 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:41.715854 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:41.758429 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a388eb-6c09-4a3a-8e53-6aaa2706a274 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:41.790945 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407329a3-85ba-4820-9ce5-b23bef8eaab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:41.791593 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407329a3-85ba-4820-9ce5-b23bef8eaab1 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:41.791876 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407329a3-85ba-4820-9ce5-b23bef8eaab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:41.792147 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407329a3-85ba-4820-9ce5-b23bef8eaab1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:41.792961 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407329a3-85ba-4820-9ce5-b23bef8eaab1 None admin] RBAC: Adding query filter params (role.id=d5899b9e6b9940aa9837652e51a23908, scope.project.id=c5260c7fd3614e9e93b167c86a94260e, user.id=de1632555cb345f2b9213da1925c02f4) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:41.793289 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407329a3-85ba-4820-9ce5-b23bef8eaab1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:41.794699 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407329a3-85ba-4820-9ce5-b23bef8eaab1 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:41.800333 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 566/1106] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:16:41 2022] GET /identity/v3/role_assignments?user.id=de1632555cb345f2b9213da1925c02f4&scope.project.id=c5260c7fd3614e9e93b167c86a94260e&role.id=d5899b9e6b9940aa9837652e51a23908 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:16:42.910746 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c64a6f-643b-4871-8fc0-74747da37599 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:42.911292 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c64a6f-643b-4871-8fc0-74747da37599 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:42.911292 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c64a6f-643b-4871-8fc0-74747da37599 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:42.912719 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 541/1107] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:16:42.917919 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce742eb8-bd0a-47a6-aebc-9fc06727c631 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:42.918310 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce742eb8-bd0a-47a6-aebc-9fc06727c631 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:42.918474 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce742eb8-bd0a-47a6-aebc-9fc06727c631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:42.935295 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-ce742eb8-bd0a-47a6-aebc-9fc06727c631 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:42.936507 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-ce742eb8-bd0a-47a6-aebc-9fc06727c631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:43.002373 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 567/1108] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:16:43.027309 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c64a6f-643b-4871-8fc0-74747da37599 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:43.130297 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c64a6f-643b-4871-8fc0-74747da37599 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:43.165589 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5f318-108f-4b5e-97b4-b29b95c7e911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:43.166299 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5f318-108f-4b5e-97b4-b29b95c7e911 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:43.166581 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5f318-108f-4b5e-97b4-b29b95c7e911 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:43.166852 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5f318-108f-4b5e-97b4-b29b95c7e911 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:43.168542 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e5f318-108f-4b5e-97b4-b29b95c7e911 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:43.169950 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e5f318-108f-4b5e-97b4-b29b95c7e911 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:43.171548 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 542/1109] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:16:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:43.177178 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce742eb8-bd0a-47a6-aebc-9fc06727c631 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:43.224340 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:43.272975 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce742eb8-bd0a-47a6-aebc-9fc06727c631 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:43.305358 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd082576-ae9e-44dc-a7e7-86fdd635aadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:43.306005 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd082576-ae9e-44dc-a7e7-86fdd635aadb None admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:43.306287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd082576-ae9e-44dc-a7e7-86fdd635aadb None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:43.306560 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd082576-ae9e-44dc-a7e7-86fdd635aadb None admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:43.307269 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd082576-ae9e-44dc-a7e7-86fdd635aadb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:43.309304 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd082576-ae9e-44dc-a7e7-86fdd635aadb None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:43.390020 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 568/1110] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Jul 7 15:16:43 2022] POST /identity/v3/users => generated 286 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:16:45.437774 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c7fc6-ffbc-427b-b360-0641d19d550b None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:45.438375 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c7fc6-ffbc-427b-b360-0641d19d550b None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:45.438375 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c7fc6-ffbc-427b-b360-0641d19d550b None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:45.439753 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 543/1111] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:16:45.447429 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5fb5f-9e9f-4f64-a7b9-734fc2cf95c5 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:45.447930 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5fb5f-9e9f-4f64-a7b9-734fc2cf95c5 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:45.448030 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5fb5f-9e9f-4f64-a7b9-734fc2cf95c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:45.466678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-abd5fb5f-9e9f-4f64-a7b9-734fc2cf95c5 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:45.467900 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-abd5fb5f-9e9f-4f64-a7b9-734fc2cf95c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:45.535996 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 569/1112] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:16:45.559128 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c7fc6-ffbc-427b-b360-0641d19d550b None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:45.620185 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:45.660936 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c7fc6-ffbc-427b-b360-0641d19d550b None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:45.693113 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadc2d69-a6ec-4e21-9e5c-c35d4307fe86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:45.693893 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc2d69-a6ec-4e21-9e5c-c35d4307fe86 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:45.694176 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc2d69-a6ec-4e21-9e5c-c35d4307fe86 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:45.694454 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc2d69-a6ec-4e21-9e5c-c35d4307fe86 None admin] PATH_INFO: `/v3/roles/c82203e4734346719cafb16230f45942` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:45.695962 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadc2d69-a6ec-4e21-9e5c-c35d4307fe86 None admin] RBAC: Authorizing `identity:get_role(role_id=c82203e4734346719cafb16230f45942)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:45.697315 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadc2d69-a6ec-4e21-9e5c-c35d4307fe86 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:45.698665 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 544/1113] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:16:45 2022] GET /identity/v3/roles/c82203e4734346719cafb16230f45942 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:45.704910 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd5fb5f-9e9f-4f64-a7b9-734fc2cf95c5 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:45.737757 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:45.807775 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd5fb5f-9e9f-4f64-a7b9-734fc2cf95c5 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:45.841007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97abf0d-3e6b-4d4c-8f0d-c0c54839889c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:45.841858 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97abf0d-3e6b-4d4c-8f0d-c0c54839889c None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:45.842139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97abf0d-3e6b-4d4c-8f0d-c0c54839889c None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:45.842410 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97abf0d-3e6b-4d4c-8f0d-c0c54839889c None admin] PATH_INFO: `/v3/users/0b8e84ecaaea419faa4b6a7b82c7a677` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:45.855122 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97abf0d-3e6b-4d4c-8f0d-c0c54839889c None admin] RBAC: Authorizing `identity:get_user(user_id=0b8e84ecaaea419faa4b6a7b82c7a677)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:45.856608 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97abf0d-3e6b-4d4c-8f0d-c0c54839889c None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:45.858226 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 570/1114] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:16:45 2022] GET /identity/v3/users/0b8e84ecaaea419faa4b6a7b82c7a677 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:45.864009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadc2d69-a6ec-4e21-9e5c-c35d4307fe86 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:45.960550 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadc2d69-a6ec-4e21-9e5c-c35d4307fe86 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:45.986215 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:45.993856 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9054cb1-5348-4034-a1f2-9553a1ce2ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:45.996714 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9054cb1-5348-4034-a1f2-9553a1ce2ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:45.996714 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9054cb1-5348-4034-a1f2-9553a1ce2ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:45.996714 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9054cb1-5348-4034-a1f2-9553a1ce2ff4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:46.060091 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9054cb1-5348-4034-a1f2-9553a1ce2ff4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:46.061505 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9054cb1-5348-4034-a1f2-9553a1ce2ff4 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:46.163236 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 545/1115] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:16:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 07 15:16:46.169146 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97abf0d-3e6b-4d4c-8f0d-c0c54839889c None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:46.259030 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:46.267840 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97abf0d-3e6b-4d4c-8f0d-c0c54839889c None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:46.300650 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a3afb5-2d62-4000-b12b-80cdf2f0c630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:46.301497 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3afb5-2d62-4000-b12b-80cdf2f0c630 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:46.301779 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3afb5-2d62-4000-b12b-80cdf2f0c630 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:46.302053 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3afb5-2d62-4000-b12b-80cdf2f0c630 None admin] PATH_INFO: `/v3/projects/22fae5dcc60a4216bc1a506128557b5f` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:46.303211 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a3afb5-2d62-4000-b12b-80cdf2f0c630 None admin] RBAC: Authorizing `identity:get_project(project_id=22fae5dcc60a4216bc1a506128557b5f)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:46.304661 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a3afb5-2d62-4000-b12b-80cdf2f0c630 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:46.306323 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 571/1116] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:16:46 2022] GET /identity/v3/projects/22fae5dcc60a4216bc1a506128557b5f => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:46.312554 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9054cb1-5348-4034-a1f2-9553a1ce2ff4 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:46.339203 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:46.428262 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9054cb1-5348-4034-a1f2-9553a1ce2ff4 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:46.467013 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c822ce-307b-4b60-a316-ebbd27eb4e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:46.467679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c822ce-307b-4b60-a316-ebbd27eb4e94 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:46.467962 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c822ce-307b-4b60-a316-ebbd27eb4e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:46.468234 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c822ce-307b-4b60-a316-ebbd27eb4e94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:46.469053 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c822ce-307b-4b60-a316-ebbd27eb4e94 None admin] RBAC: Adding query filter params (role.id=c82203e4734346719cafb16230f45942, scope.project.id=22fae5dcc60a4216bc1a506128557b5f, user.id=0b8e84ecaaea419faa4b6a7b82c7a677) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:46.469361 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c822ce-307b-4b60-a316-ebbd27eb4e94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:46.470759 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c822ce-307b-4b60-a316-ebbd27eb4e94 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:46.476279 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 546/1117] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:16:46 2022] GET /identity/v3/role_assignments?user.id=0b8e84ecaaea419faa4b6a7b82c7a677&scope.project.id=22fae5dcc60a4216bc1a506128557b5f&role.id=c82203e4734346719cafb16230f45942 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:48.441298 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a3394-c290-4691-8d6a-d5b03098644d None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:48.441817 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a3394-c290-4691-8d6a-d5b03098644d None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:48.441927 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a3394-c290-4691-8d6a-d5b03098644d None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:48.443194 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 572/1118] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:16:48.448305 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2f9ca-c48f-4a1b-a15d-7bb141d7816b None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:48.448722 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2f9ca-c48f-4a1b-a15d-7bb141d7816b None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:48.448857 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2f9ca-c48f-4a1b-a15d-7bb141d7816b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:48.465061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-dfe2f9ca-c48f-4a1b-a15d-7bb141d7816b None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:48.466221 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-dfe2f9ca-c48f-4a1b-a15d-7bb141d7816b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:48.529961 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 547/1119] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:16:48.552039 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a3394-c290-4691-8d6a-d5b03098644d None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:48.649145 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a3394-c290-4691-8d6a-d5b03098644d None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:48.681420 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785b951-9c22-4cd1-8cdb-684438d52141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:48.682254 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785b951-9c22-4cd1-8cdb-684438d52141 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:48.682536 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785b951-9c22-4cd1-8cdb-684438d52141 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:48.682811 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785b951-9c22-4cd1-8cdb-684438d52141 None admin] PATH_INFO: `/v3/roles/c82203e4734346719cafb16230f45942` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:48.684794 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e785b951-9c22-4cd1-8cdb-684438d52141 None admin] RBAC: Authorizing `identity:get_role(role_id=c82203e4734346719cafb16230f45942)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:48.686151 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e785b951-9c22-4cd1-8cdb-684438d52141 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:48.687475 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 573/1120] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:16:48 2022] GET /identity/v3/roles/c82203e4734346719cafb16230f45942 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:48.693094 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe2f9ca-c48f-4a1b-a15d-7bb141d7816b None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:48.716247 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:48.788536 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe2f9ca-c48f-4a1b-a15d-7bb141d7816b None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:48.821760 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d87bb62-eb6e-4c7e-b432-a0bde2fb432f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:48.822603 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87bb62-eb6e-4c7e-b432-a0bde2fb432f None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:48.822878 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87bb62-eb6e-4c7e-b432-a0bde2fb432f None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:48.823147 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87bb62-eb6e-4c7e-b432-a0bde2fb432f None admin] PATH_INFO: `/v3/users/0b8e84ecaaea419faa4b6a7b82c7a677` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:48.824407 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d87bb62-eb6e-4c7e-b432-a0bde2fb432f None admin] RBAC: Authorizing `identity:get_user(user_id=0b8e84ecaaea419faa4b6a7b82c7a677)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:48.825835 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d87bb62-eb6e-4c7e-b432-a0bde2fb432f None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:48.827488 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 548/1121] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:16:48 2022] GET /identity/v3/users/0b8e84ecaaea419faa4b6a7b82c7a677 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:48.833070 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785b951-9c22-4cd1-8cdb-684438d52141 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:48.861178 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:48.929303 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785b951-9c22-4cd1-8cdb-684438d52141 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:48.962401 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd8f77-766f-42a4-9a3b-776a38f5fcbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:48.963195 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd8f77-766f-42a4-9a3b-776a38f5fcbc None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:48.963479 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd8f77-766f-42a4-9a3b-776a38f5fcbc None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:48.963784 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd8f77-766f-42a4-9a3b-776a38f5fcbc None admin] PATH_INFO: `/v3/projects/22fae5dcc60a4216bc1a506128557b5f` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:48.965462 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cd8f77-766f-42a4-9a3b-776a38f5fcbc None admin] RBAC: Authorizing `identity:get_project(project_id=22fae5dcc60a4216bc1a506128557b5f)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:48.966927 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cd8f77-766f-42a4-9a3b-776a38f5fcbc None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:48.968649 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 574/1122] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:16:48 2022] GET /identity/v3/projects/22fae5dcc60a4216bc1a506128557b5f => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:48.974701 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d87bb62-eb6e-4c7e-b432-a0bde2fb432f None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:49.062638 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:49.071856 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d87bb62-eb6e-4c7e-b432-a0bde2fb432f None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:49.104965 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a6136-37a4-4e9d-ac7b-c7deb6fcc474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:49.106513 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a6136-37a4-4e9d-ac7b-c7deb6fcc474 None admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:49.106795 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a6136-37a4-4e9d-ac7b-c7deb6fcc474 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:49.107065 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a6136-37a4-4e9d-ac7b-c7deb6fcc474 None admin] PATH_INFO: `/v3/projects/22fae5dcc60a4216bc1a506128557b5f/users/0b8e84ecaaea419faa4b6a7b82c7a677/roles/c82203e4734346719cafb16230f45942` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:49.109215 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14a6136-37a4-4e9d-ac7b-c7deb6fcc474 None admin] RBAC: Authorizing `identity:create_grant(project_id=22fae5dcc60a4216bc1a506128557b5f, user_id=0b8e84ecaaea419faa4b6a7b82c7a677, role_id=c82203e4734346719cafb16230f45942)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:49.110622 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14a6136-37a4-4e9d-ac7b-c7deb6fcc474 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:49.172019 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 549/1123] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Jul 7 15:16:48 2022] PUT /identity/v3/projects/22fae5dcc60a4216bc1a506128557b5f/users/0b8e84ecaaea419faa4b6a7b82c7a677/roles/c82203e4734346719cafb16230f45942 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:16:51.201138 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1e894-4b35-4ad4-8185-af6aa04347c4 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:51.201650 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1e894-4b35-4ad4-8185-af6aa04347c4 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:51.201650 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1e894-4b35-4ad4-8185-af6aa04347c4 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:51.203010 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 575/1124] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:16:51.208163 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bfa72-ab8a-4321-ad0b-e7a1ece80a65 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:51.208585 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bfa72-ab8a-4321-ad0b-e7a1ece80a65 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:51.208712 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bfa72-ab8a-4321-ad0b-e7a1ece80a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:51.225893 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-bf4bfa72-ab8a-4321-ad0b-e7a1ece80a65 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:51.227076 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-bf4bfa72-ab8a-4321-ad0b-e7a1ece80a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:51.290700 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 550/1125] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:16:51.313793 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c1e894-4b35-4ad4-8185-af6aa04347c4 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:51.406829 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:51.413375 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c1e894-4b35-4ad4-8185-af6aa04347c4 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:51.445386 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d9e746-98c2-4632-9384-73f827aa51d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:51.446226 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9e746-98c2-4632-9384-73f827aa51d0 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:51.446510 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9e746-98c2-4632-9384-73f827aa51d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:51.446788 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9e746-98c2-4632-9384-73f827aa51d0 None admin] PATH_INFO: `/v3/roles/c82203e4734346719cafb16230f45942` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:51.448285 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d9e746-98c2-4632-9384-73f827aa51d0 None admin] RBAC: Authorizing `identity:get_role(role_id=c82203e4734346719cafb16230f45942)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:51.450242 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d9e746-98c2-4632-9384-73f827aa51d0 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:51.452562 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 576/1126] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:16:51 2022] GET /identity/v3/roles/c82203e4734346719cafb16230f45942 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:51.459778 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4bfa72-ab8a-4321-ad0b-e7a1ece80a65 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:51.536832 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:51.554590 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4bfa72-ab8a-4321-ad0b-e7a1ece80a65 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:51.587525 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b759c-b513-4689-b06c-10962691912f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:51.588400 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b759c-b513-4689-b06c-10962691912f None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:51.588676 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b759c-b513-4689-b06c-10962691912f None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:51.588950 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b759c-b513-4689-b06c-10962691912f None admin] PATH_INFO: `/v3/users/0b8e84ecaaea419faa4b6a7b82c7a677` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:51.590712 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7b759c-b513-4689-b06c-10962691912f None admin] RBAC: Authorizing `identity:get_user(user_id=0b8e84ecaaea419faa4b6a7b82c7a677)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:51.592146 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7b759c-b513-4689-b06c-10962691912f None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:51.593824 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 551/1127] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:16:51 2022] GET /identity/v3/users/0b8e84ecaaea419faa4b6a7b82c7a677 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:51.599767 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d9e746-98c2-4632-9384-73f827aa51d0 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:51.695932 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d9e746-98c2-4632-9384-73f827aa51d0 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:51.729238 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a11c03e-d2e8-4e5b-adca-ff6a2a1aea38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:51.729876 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11c03e-d2e8-4e5b-adca-ff6a2a1aea38 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:51.730199 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11c03e-d2e8-4e5b-adca-ff6a2a1aea38 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:51.730474 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11c03e-d2e8-4e5b-adca-ff6a2a1aea38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:51.759656 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:51.795146 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a11c03e-d2e8-4e5b-adca-ff6a2a1aea38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:51.796544 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a11c03e-d2e8-4e5b-adca-ff6a2a1aea38 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:51.891273 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 577/1128] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:16:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 07 15:16:51.897147 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b759c-b513-4689-b06c-10962691912f None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:51.996186 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b759c-b513-4689-b06c-10962691912f None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:52.028904 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f6c3af-38b3-4770-b422-33372e06d975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:52.029753 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6c3af-38b3-4770-b422-33372e06d975 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:52.030033 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6c3af-38b3-4770-b422-33372e06d975 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:52.030307 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6c3af-38b3-4770-b422-33372e06d975 None admin] PATH_INFO: `/v3/projects/22fae5dcc60a4216bc1a506128557b5f` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:52.031464 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f6c3af-38b3-4770-b422-33372e06d975 None admin] RBAC: Authorizing `identity:get_project(project_id=22fae5dcc60a4216bc1a506128557b5f)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:52.032907 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f6c3af-38b3-4770-b422-33372e06d975 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:52.034585 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 552/1129] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:16:51 2022] GET /identity/v3/projects/22fae5dcc60a4216bc1a506128557b5f => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:52.040476 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a11c03e-d2e8-4e5b-adca-ff6a2a1aea38 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:52.100226 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:52.136099 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a11c03e-d2e8-4e5b-adca-ff6a2a1aea38 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:52.168789 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e0677-bb0d-46a7-8aab-b3573f56178c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:52.169420 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e0677-bb0d-46a7-8aab-b3573f56178c None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:52.169701 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e0677-bb0d-46a7-8aab-b3573f56178c None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:52.169977 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e0677-bb0d-46a7-8aab-b3573f56178c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:52.170797 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e0677-bb0d-46a7-8aab-b3573f56178c None admin] RBAC: Adding query filter params (role.id=c82203e4734346719cafb16230f45942, scope.project.id=22fae5dcc60a4216bc1a506128557b5f, user.id=0b8e84ecaaea419faa4b6a7b82c7a677) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:52.171107 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e0677-bb0d-46a7-8aab-b3573f56178c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:52.172551 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e0677-bb0d-46a7-8aab-b3573f56178c None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:52.178148 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 578/1130] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:16:52 2022] GET /identity/v3/role_assignments?user.id=0b8e84ecaaea419faa4b6a7b82c7a677&scope.project.id=22fae5dcc60a4216bc1a506128557b5f&role.id=c82203e4734346719cafb16230f45942 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:54.183343 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10c81c-e55f-4492-9200-278a81703f88 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:54.183884 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10c81c-e55f-4492-9200-278a81703f88 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:54.184009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10c81c-e55f-4492-9200-278a81703f88 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:54.185277 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 553/1131] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:16:54.190412 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5744d6-6c0b-4f07-b457-d92e09a7e951 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:54.190836 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5744d6-6c0b-4f07-b457-d92e09a7e951 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:54.190964 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5744d6-6c0b-4f07-b457-d92e09a7e951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:54.207260 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-bf5744d6-6c0b-4f07-b457-d92e09a7e951 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:54.208474 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-bf5744d6-6c0b-4f07-b457-d92e09a7e951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:54.273567 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 579/1132] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:16:54.297636 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f10c81c-e55f-4492-9200-278a81703f88 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:54.315402 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:54.397018 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f10c81c-e55f-4492-9200-278a81703f88 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:54.429774 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0944b48c-e485-4648-a4f5-0b4774c38c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:54.430541 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944b48c-e485-4648-a4f5-0b4774c38c03 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:54.430824 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944b48c-e485-4648-a4f5-0b4774c38c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:54.431099 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944b48c-e485-4648-a4f5-0b4774c38c03 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:54.432238 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0944b48c-e485-4648-a4f5-0b4774c38c03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:54.433687 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0944b48c-e485-4648-a4f5-0b4774c38c03 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:54.435284 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 554/1133] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:16:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:54.440913 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5744d6-6c0b-4f07-b457-d92e09a7e951 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:54.497684 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:54.538599 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5744d6-6c0b-4f07-b457-d92e09a7e951 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:54.570720 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44661e8c-8315-4b51-ae43-511df6dc26d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:54.571362 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44661e8c-8315-4b51-ae43-511df6dc26d8 None admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:54.571673 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44661e8c-8315-4b51-ae43-511df6dc26d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:54.571949 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44661e8c-8315-4b51-ae43-511df6dc26d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:54.572641 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44661e8c-8315-4b51-ae43-511df6dc26d8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:54.574599 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44661e8c-8315-4b51-ae43-511df6dc26d8 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:54.647776 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 580/1134] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Jul 7 15:16:54 2022] POST /identity/v3/projects => generated 311 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:16:56.673537 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e23f73-63d4-4c81-a0fa-f5652bde2f58 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:56.674090 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e23f73-63d4-4c81-a0fa-f5652bde2f58 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:56.674090 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e23f73-63d4-4c81-a0fa-f5652bde2f58 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:56.675447 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 555/1135] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:16:56.680717 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c378b-ba20-4f98-8929-b549fc67d0b1 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:56.681115 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c378b-ba20-4f98-8929-b549fc67d0b1 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:56.681268 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c378b-ba20-4f98-8929-b549fc67d0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:56.698217 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-777c378b-ba20-4f98-8929-b549fc67d0b1 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:56.699381 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-777c378b-ba20-4f98-8929-b549fc67d0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:56.763536 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 581/1136] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:16:56.787441 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e23f73-63d4-4c81-a0fa-f5652bde2f58 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:56.870205 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:56.886924 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e23f73-63d4-4c81-a0fa-f5652bde2f58 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:56.920634 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d29d9-9cff-4871-b570-86656115a978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:56.921406 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d29d9-9cff-4871-b570-86656115a978 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:56.921684 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d29d9-9cff-4871-b570-86656115a978 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:56.921955 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d29d9-9cff-4871-b570-86656115a978 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:56.923096 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292d29d9-9cff-4871-b570-86656115a978 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:56.924644 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292d29d9-9cff-4871-b570-86656115a978 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:56.926257 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 556/1137] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:16:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:56.931833 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c378b-ba20-4f98-8929-b549fc67d0b1 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:56.972835 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:57.027784 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c378b-ba20-4f98-8929-b549fc67d0b1 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:57.061367 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed10e4a0-a078-4c9a-ad74-6e9a67faa0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:57.062025 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed10e4a0-a078-4c9a-ad74-6e9a67faa0c9 None admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:57.062306 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed10e4a0-a078-4c9a-ad74-6e9a67faa0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:57.062579 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed10e4a0-a078-4c9a-ad74-6e9a67faa0c9 None admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:57.063346 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed10e4a0-a078-4c9a-ad74-6e9a67faa0c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:57.064806 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed10e4a0-a078-4c9a-ad74-6e9a67faa0c9 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:57.178266 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 582/1138] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Jul 7 15:16:56 2022] POST /identity/v3/users => generated 286 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:16:59.208527 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50a336-1c11-41e7-8d4e-d20593a4667a None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:59.209036 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50a336-1c11-41e7-8d4e-d20593a4667a None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:59.209155 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50a336-1c11-41e7-8d4e-d20593a4667a None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:59.210425 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 557/1139] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:16:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:16:59.216408 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d1c10-ddae-4746-b342-fac3935f0250 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:59.216812 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d1c10-ddae-4746-b342-fac3935f0250 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:59.216904 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d1c10-ddae-4746-b342-fac3935f0250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:59.233539 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-2c0d1c10-ddae-4746-b342-fac3935f0250 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:16:59.234734 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-2c0d1c10-ddae-4746-b342-fac3935f0250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:16:59.299501 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 583/1140] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:16:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:16:59.322328 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e50a336-1c11-41e7-8d4e-d20593a4667a None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:59.420280 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e50a336-1c11-41e7-8d4e-d20593a4667a None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:59.452040 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb87df-4b71-4716-9318-3d018e7ca720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:59.452804 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb87df-4b71-4716-9318-3d018e7ca720 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:59.453084 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb87df-4b71-4716-9318-3d018e7ca720 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:59.453355 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb87df-4b71-4716-9318-3d018e7ca720 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:59.466855 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bb87df-4b71-4716-9318-3d018e7ca720 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:16:59.476556 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-49bb87df-4b71-4716-9318-3d018e7ca720 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:16:59.476556 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:16:59.476556 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:16:59.476556 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:16:59.476556 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:16:59.476556 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:16:59.476556 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:16:59.476556 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:16:59.476556 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:16:59.476556 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:16:59.476556 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:16:59.476556 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:16:59.476556 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:16:59.476556 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:16:59.476556 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:16:59.476556 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:16:59.477749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:16:59.477749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:16:59.477749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:16:59.477749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:16:59.477749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:16:59.477749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:16:59.477749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:16:59.477749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:16:59.477749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:16:59.477749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:16:59.477749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:16:59.477749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:16:59.477749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:16:59.477749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:16:59.477749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:16:59.477749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:16:59.478870 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:16:59.478870 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:16:59.478870 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:16:59.478870 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:16:59.478870 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:16:59.478870 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:16:59.478870 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:16:59.478870 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:16:59.478870 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:16:59.478870 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:16:59.478870 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:16:59.478870 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:16:59.478870 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 558/1141] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Jul 7 15:16:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:16:59.484517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d1c10-ddae-4746-b342-fac3935f0250 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:59.510910 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:59.579504 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d1c10-ddae-4746-b342-fac3935f0250 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:59.613525 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210c2590-7b9c-48cf-be07-489ff5d82dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:59.614170 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c2590-7b9c-48cf-be07-489ff5d82dfa None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:59.614448 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c2590-7b9c-48cf-be07-489ff5d82dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:59.614718 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c2590-7b9c-48cf-be07-489ff5d82dfa None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:59.615453 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210c2590-7b9c-48cf-be07-489ff5d82dfa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:16:59.615784 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210c2590-7b9c-48cf-be07-489ff5d82dfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:59.617139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210c2590-7b9c-48cf-be07-489ff5d82dfa None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:59.624814 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 584/1142] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:16:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:59.630622 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb87df-4b71-4716-9318-3d018e7ca720 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:59.646101 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:59.728722 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb87df-4b71-4716-9318-3d018e7ca720 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:59.765710 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2363048-55a1-48a1-8481-308ca135bf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:59.766565 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2363048-55a1-48a1-8481-308ca135bf45 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:59.766841 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2363048-55a1-48a1-8481-308ca135bf45 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:59.767110 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2363048-55a1-48a1-8481-308ca135bf45 None admin] PATH_INFO: `/v3/users/2c236e0e9a90491884d9f9c3feac5a8c` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:59.780644 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2363048-55a1-48a1-8481-308ca135bf45 None admin] RBAC: Authorizing `identity:get_user(user_id=2c236e0e9a90491884d9f9c3feac5a8c)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:59.782067 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2363048-55a1-48a1-8481-308ca135bf45 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:16:59.783741 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 559/1143] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:16:59 2022] GET /identity/v3/users/2c236e0e9a90491884d9f9c3feac5a8c => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:16:59.789633 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210c2590-7b9c-48cf-be07-489ff5d82dfa None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:16:59.875121 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:16:59.886809 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210c2590-7b9c-48cf-be07-489ff5d82dfa None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:16:59.919328 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc83cf-9477-4344-9567-6582eabd2259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:16:59.919993 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc83cf-9477-4344-9567-6582eabd2259 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:16:59.920332 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc83cf-9477-4344-9567-6582eabd2259 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:16:59.920606 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc83cf-9477-4344-9567-6582eabd2259 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:16:59.991066 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dc83cf-9477-4344-9567-6582eabd2259 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:16:59.992472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dc83cf-9477-4344-9567-6582eabd2259 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:17:00.087271 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:17:00.104683 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 585/1144] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:16:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 07 15:17:00.110201 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2363048-55a1-48a1-8481-308ca135bf45 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:17:00.179579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:17:00.210536 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2363048-55a1-48a1-8481-308ca135bf45 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:17:00.242903 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df5cc89-8f49-4567-8466-cc9097c04860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:17:00.243715 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5cc89-8f49-4567-8466-cc9097c04860 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:00.243997 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5cc89-8f49-4567-8466-cc9097c04860 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:00.244269 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5cc89-8f49-4567-8466-cc9097c04860 None admin] PATH_INFO: `/v3/projects/db7dc0a42eac4d81b5cdb3acfea04b91` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:00.245956 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df5cc89-8f49-4567-8466-cc9097c04860 None admin] RBAC: Authorizing `identity:get_project(project_id=db7dc0a42eac4d81b5cdb3acfea04b91)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:17:00.247361 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df5cc89-8f49-4567-8466-cc9097c04860 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:17:00.249062 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 560/1145] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:17:00 2022] GET /identity/v3/projects/db7dc0a42eac4d81b5cdb3acfea04b91 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:17:00.255140 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc83cf-9477-4344-9567-6582eabd2259 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:17:00.355510 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc83cf-9477-4344-9567-6582eabd2259 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:17:00.391171 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e2f45-fce5-4c3a-b66a-ad51590ae069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:17:00.391868 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e2f45-fce5-4c3a-b66a-ad51590ae069 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:00.392907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e2f45-fce5-4c3a-b66a-ad51590ae069 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:00.393194 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e2f45-fce5-4c3a-b66a-ad51590ae069 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:00.394027 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95e2f45-fce5-4c3a-b66a-ad51590ae069 None admin] RBAC: Adding query filter params (role.id=0e7793fbf3064037984da651879e3a94, scope.project.id=db7dc0a42eac4d81b5cdb3acfea04b91, user.id=2c236e0e9a90491884d9f9c3feac5a8c) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:17:00.394338 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95e2f45-fce5-4c3a-b66a-ad51590ae069 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:17:00.395784 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95e2f45-fce5-4c3a-b66a-ad51590ae069 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:17:00.401883 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 586/1146] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:17:00 2022] GET /identity/v3/role_assignments?user.id=2c236e0e9a90491884d9f9c3feac5a8c&scope.project.id=db7dc0a42eac4d81b5cdb3acfea04b91&role.id=0e7793fbf3064037984da651879e3a94 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:17:02.380962 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f53a4a-1aaa-4fef-afc0-f7d62c598bde None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:02.381483 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f53a4a-1aaa-4fef-afc0-f7d62c598bde None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:02.381579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f53a4a-1aaa-4fef-afc0-f7d62c598bde None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:02.382867 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 561/1147] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:17:02.388112 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af2d9e-ae53-42b1-9ee4-e0df2e6c4de4 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:02.388541 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af2d9e-ae53-42b1-9ee4-e0df2e6c4de4 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:02.388669 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af2d9e-ae53-42b1-9ee4-e0df2e6c4de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:02.405507 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-b3af2d9e-ae53-42b1-9ee4-e0df2e6c4de4 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:17:02.406694 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-b3af2d9e-ae53-42b1-9ee4-e0df2e6c4de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:17:02.452074 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:17:02.471864 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 587/1148] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:17:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:17:02.494364 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f53a4a-1aaa-4fef-afc0-f7d62c598bde None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:17:02.591698 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f53a4a-1aaa-4fef-afc0-f7d62c598bde None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:17:02.624408 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac576acc-be6a-40f7-9a14-7751027ed4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:17:02.625172 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac576acc-be6a-40f7-9a14-7751027ed4bd None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:02.625452 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac576acc-be6a-40f7-9a14-7751027ed4bd None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:02.625724 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac576acc-be6a-40f7-9a14-7751027ed4bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:02.632331 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:17:02.636542 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac576acc-be6a-40f7-9a14-7751027ed4bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:17:02.644012 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-ac576acc-be6a-40f7-9a14-7751027ed4bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:17:02.644012 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:17:02.644012 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:17:02.644012 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:17:02.644012 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:17:02.644012 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:17:02.644012 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:17:02.644012 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:17:02.644012 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:17:02.644012 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:17:02.644012 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:17:02.644012 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:17:02.644012 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:17:02.644012 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:17:02.644012 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:17:02.644012 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:17:02.645145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:17:02.645145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:17:02.645145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:17:02.645145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:17:02.645145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:17:02.645145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:17:02.645145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:17:02.645145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:17:02.645145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:17:02.645145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:17:02.645145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:17:02.645145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:17:02.645145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:17:02.645145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:17:02.645145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:17:02.645145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:17:02.646131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:17:02.646131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:17:02.646131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:17:02.646131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:17:02.646131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:17:02.646131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:17:02.646131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:17:02.646131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:17:02.646131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:17:02.646131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:17:02.646131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:17:02.646131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:17:02.646131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 562/1149] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Jul 7 15:17:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:17:02.651033 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3af2d9e-ae53-42b1-9ee4-e0df2e6c4de4 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:17:02.746505 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3af2d9e-ae53-42b1-9ee4-e0df2e6c4de4 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:17:02.779747 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef6383-3ebf-4d73-afc7-ebfd5a9a2787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:17:02.780397 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef6383-3ebf-4d73-afc7-ebfd5a9a2787 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:02.780681 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef6383-3ebf-4d73-afc7-ebfd5a9a2787 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:02.780957 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef6383-3ebf-4d73-afc7-ebfd5a9a2787 None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:02.781695 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ef6383-3ebf-4d73-afc7-ebfd5a9a2787 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:17:02.782000 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ef6383-3ebf-4d73-afc7-ebfd5a9a2787 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:17:02.783362 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ef6383-3ebf-4d73-afc7-ebfd5a9a2787 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:17:02.791125 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 588/1150] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:17:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:17:02.796867 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac576acc-be6a-40f7-9a14-7751027ed4bd None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:17:02.895444 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac576acc-be6a-40f7-9a14-7751027ed4bd None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:17:02.928551 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad782b5f-b6b5-4090-af8b-8b13279509d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:17:02.929391 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad782b5f-b6b5-4090-af8b-8b13279509d6 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:02.929668 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad782b5f-b6b5-4090-af8b-8b13279509d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:02.929937 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad782b5f-b6b5-4090-af8b-8b13279509d6 None admin] PATH_INFO: `/v3/users/2c236e0e9a90491884d9f9c3feac5a8c` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:02.931124 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad782b5f-b6b5-4090-af8b-8b13279509d6 None admin] RBAC: Authorizing `identity:get_user(user_id=2c236e0e9a90491884d9f9c3feac5a8c)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:17:02.932557 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad782b5f-b6b5-4090-af8b-8b13279509d6 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:17:02.934152 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 563/1151] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:17:02 2022] GET /identity/v3/users/2c236e0e9a90491884d9f9c3feac5a8c => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:17:02.939794 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef6383-3ebf-4d73-afc7-ebfd5a9a2787 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:17:02.985434 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:17:03.045266 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef6383-3ebf-4d73-afc7-ebfd5a9a2787 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:17:03.078535 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b5007-f056-40b1-b136-9f6849933d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:17:03.079390 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b5007-f056-40b1-b136-9f6849933d96 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:03.079701 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b5007-f056-40b1-b136-9f6849933d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:03.079981 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b5007-f056-40b1-b136-9f6849933d96 None admin] PATH_INFO: `/v3/projects/db7dc0a42eac4d81b5cdb3acfea04b91` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:03.081141 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b5007-f056-40b1-b136-9f6849933d96 None admin] RBAC: Authorizing `identity:get_project(project_id=db7dc0a42eac4d81b5cdb3acfea04b91)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:17:03.082599 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b5007-f056-40b1-b136-9f6849933d96 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:17:03.084318 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 589/1152] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:17:02 2022] GET /identity/v3/projects/db7dc0a42eac4d81b5cdb3acfea04b91 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:17:03.090213 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad782b5f-b6b5-4090-af8b-8b13279509d6 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:17:03.152777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:17:03.187863 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad782b5f-b6b5-4090-af8b-8b13279509d6 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:17:03.220233 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965315a-11b5-4b71-b68e-186d6cd7af22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:17:03.221102 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965315a-11b5-4b71-b68e-186d6cd7af22 None admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:03.221380 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965315a-11b5-4b71-b68e-186d6cd7af22 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:03.221651 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965315a-11b5-4b71-b68e-186d6cd7af22 None admin] PATH_INFO: `/v3/projects/db7dc0a42eac4d81b5cdb3acfea04b91/users/2c236e0e9a90491884d9f9c3feac5a8c/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:03.223845 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8965315a-11b5-4b71-b68e-186d6cd7af22 None admin] RBAC: Authorizing `identity:create_grant(project_id=db7dc0a42eac4d81b5cdb3acfea04b91, user_id=2c236e0e9a90491884d9f9c3feac5a8c, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:17:03.225238 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8965315a-11b5-4b71-b68e-186d6cd7af22 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:17:03.305655 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 564/1153] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Jul 7 15:17:03 2022] PUT /identity/v3/projects/db7dc0a42eac4d81b5cdb3acfea04b91/users/2c236e0e9a90491884d9f9c3feac5a8c/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:17:05.272350 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14714da4-9d0c-4f57-ba46-ce58068f6e5c None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:05.272849 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14714da4-9d0c-4f57-ba46-ce58068f6e5c None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:05.272849 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14714da4-9d0c-4f57-ba46-ce58068f6e5c None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:05.274143 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 590/1154] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:17:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:17:05.279808 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5dc1e-b2f2-45b0-9158-5359d5507c1f None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:05.280221 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5dc1e-b2f2-45b0-9158-5359d5507c1f None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:05.280359 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5dc1e-b2f2-45b0-9158-5359d5507c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:05.297157 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-4ff5dc1e-b2f2-45b0-9158-5359d5507c1f None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:17:05.298350 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-4ff5dc1e-b2f2-45b0-9158-5359d5507c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:17:05.361791 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 565/1155] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:17:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:17:05.385340 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14714da4-9d0c-4f57-ba46-ce58068f6e5c None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:17:05.484356 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14714da4-9d0c-4f57-ba46-ce58068f6e5c None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:17:05.496980 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:17:05.517430 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc6e9c-1394-4335-8c41-9d63760f5221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:17:05.521391 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc6e9c-1394-4335-8c41-9d63760f5221 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:05.521391 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc6e9c-1394-4335-8c41-9d63760f5221 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:05.521391 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc6e9c-1394-4335-8c41-9d63760f5221 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:05.529136 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bc6e9c-1394-4335-8c41-9d63760f5221 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:17:05.536576 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-e7bc6e9c-1394-4335-8c41-9d63760f5221 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:17:05.536576 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:17:05.536576 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:17:05.536576 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:17:05.536576 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:17:05.536576 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:17:05.536576 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:17:05.536576 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:17:05.536576 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:17:05.536576 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:17:05.536576 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:17:05.536576 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:17:05.536576 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:17:05.536576 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:17:05.536576 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:17:05.536576 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:17:05.537687 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:17:05.537687 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:17:05.537687 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:17:05.537687 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:17:05.537687 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:17:05.537687 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:17:05.537687 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:17:05.537687 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:17:05.537687 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:17:05.537687 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:17:05.537687 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:17:05.537687 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:17:05.537687 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:17:05.537687 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:17:05.537687 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:17:05.537687 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:17:05.538753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:17:05.538753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:17:05.538753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:17:05.538753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:17:05.538753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:17:05.538753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:17:05.538753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:17:05.538753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:17:05.538753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:17:05.538753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:17:05.538753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:17:05.538753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:17:05.538753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 591/1156] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Jul 7 15:17:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:17:05.545512 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5dc1e-b2f2-45b0-9158-5359d5507c1f None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:17:05.599578 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:17:05.641952 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5dc1e-b2f2-45b0-9158-5359d5507c1f None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:17:05.674776 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02795c3c-e7e3-4ae7-ba44-71789091739a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:17:05.675434 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02795c3c-e7e3-4ae7-ba44-71789091739a None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:05.675756 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02795c3c-e7e3-4ae7-ba44-71789091739a None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:05.676037 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02795c3c-e7e3-4ae7-ba44-71789091739a None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:05.676783 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02795c3c-e7e3-4ae7-ba44-71789091739a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:17:05.677092 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02795c3c-e7e3-4ae7-ba44-71789091739a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:17:05.678446 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02795c3c-e7e3-4ae7-ba44-71789091739a None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:17:05.687059 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 566/1157] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:17:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:17:05.692740 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc6e9c-1394-4335-8c41-9d63760f5221 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:17:05.789978 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc6e9c-1394-4335-8c41-9d63760f5221 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:17:05.821783 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2e676-1c6f-4254-87b3-79705b1a378b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:17:05.822623 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2e676-1c6f-4254-87b3-79705b1a378b None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:05.822902 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2e676-1c6f-4254-87b3-79705b1a378b None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:05.823707 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2e676-1c6f-4254-87b3-79705b1a378b None admin] PATH_INFO: `/v3/users/2c236e0e9a90491884d9f9c3feac5a8c` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:05.824864 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f2e676-1c6f-4254-87b3-79705b1a378b None admin] RBAC: Authorizing `identity:get_user(user_id=2c236e0e9a90491884d9f9c3feac5a8c)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:17:05.826268 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f2e676-1c6f-4254-87b3-79705b1a378b None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:17:05.827864 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 592/1158] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:17:05 2022] GET /identity/v3/users/2c236e0e9a90491884d9f9c3feac5a8c => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:17:05.833326 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02795c3c-e7e3-4ae7-ba44-71789091739a None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:17:05.930174 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02795c3c-e7e3-4ae7-ba44-71789091739a None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:17:05.946335 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:17:05.962936 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ffecb9-619a-4567-ac37-5caa45391a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:17:05.964166 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ffecb9-619a-4567-ac37-5caa45391a76 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:05.964464 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ffecb9-619a-4567-ac37-5caa45391a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:05.965212 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ffecb9-619a-4567-ac37-5caa45391a76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:06.030521 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ffecb9-619a-4567-ac37-5caa45391a76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:17:06.031978 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ffecb9-619a-4567-ac37-5caa45391a76 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:17:06.126204 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 567/1159] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:17:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 07 15:17:06.131992 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2e676-1c6f-4254-87b3-79705b1a378b None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:17:06.172027 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:17:06.227637 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2e676-1c6f-4254-87b3-79705b1a378b None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:17:06.259895 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d79665-9e2c-497b-b9d2-143391d75262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:17:06.261310 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d79665-9e2c-497b-b9d2-143391d75262 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:06.261589 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d79665-9e2c-497b-b9d2-143391d75262 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:06.261864 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d79665-9e2c-497b-b9d2-143391d75262 None admin] PATH_INFO: `/v3/projects/db7dc0a42eac4d81b5cdb3acfea04b91` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:06.263024 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d79665-9e2c-497b-b9d2-143391d75262 None admin] RBAC: Authorizing `identity:get_project(project_id=db7dc0a42eac4d81b5cdb3acfea04b91)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:17:06.264531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d79665-9e2c-497b-b9d2-143391d75262 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:17:06.266190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 593/1160] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:17:06 2022] GET /identity/v3/projects/db7dc0a42eac4d81b5cdb3acfea04b91 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:17:06.272035 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ffecb9-619a-4567-ac37-5caa45391a76 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:17:06.289652 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:17:06.368537 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ffecb9-619a-4567-ac37-5caa45391a76 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:17:06.401289 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44748b63-e335-4816-a8a1-939c6ebf1774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:17:06.401868 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44748b63-e335-4816-a8a1-939c6ebf1774 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:06.402150 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44748b63-e335-4816-a8a1-939c6ebf1774 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:06.402446 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44748b63-e335-4816-a8a1-939c6ebf1774 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:06.403244 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44748b63-e335-4816-a8a1-939c6ebf1774 None admin] RBAC: Adding query filter params (role.id=0e7793fbf3064037984da651879e3a94, scope.project.id=db7dc0a42eac4d81b5cdb3acfea04b91, user.id=2c236e0e9a90491884d9f9c3feac5a8c) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:17:06.403552 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44748b63-e335-4816-a8a1-939c6ebf1774 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:17:06.404981 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44748b63-e335-4816-a8a1-939c6ebf1774 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:17:06.410619 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 568/1161] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:17:06 2022] GET /identity/v3/role_assignments?user.id=2c236e0e9a90491884d9f9c3feac5a8c&scope.project.id=db7dc0a42eac4d81b5cdb3acfea04b91&role.id=0e7793fbf3064037984da651879e3a94 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:17:08.417256 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82d970-4b9b-4d43-b0ce-feb4e8e7f735 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:08.417782 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82d970-4b9b-4d43-b0ce-feb4e8e7f735 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:08.417782 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82d970-4b9b-4d43-b0ce-feb4e8e7f735 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:08.419154 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 594/1162] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:17:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:17:08.424510 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490ab27-abd1-4e32-abdb-205ca0ac3770 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:08.424935 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490ab27-abd1-4e32-abdb-205ca0ac3770 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:08.425068 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490ab27-abd1-4e32-abdb-205ca0ac3770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:08.441349 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-9490ab27-abd1-4e32-abdb-205ca0ac3770 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:17:08.442528 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-9490ab27-abd1-4e32-abdb-205ca0ac3770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:17:08.506863 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 569/1163] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:17:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:17:08.529181 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc82d970-4b9b-4d43-b0ce-feb4e8e7f735 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:17:08.627113 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc82d970-4b9b-4d43-b0ce-feb4e8e7f735 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:17:08.636496 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:17:08.660527 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6c68f-27ae-47cc-baa4-852988b5cd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:17:08.661155 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6c68f-27ae-47cc-baa4-852988b5cd5e None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:08.661436 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6c68f-27ae-47cc-baa4-852988b5cd5e None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:08.661709 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6c68f-27ae-47cc-baa4-852988b5cd5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:08.727745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f6c68f-27ae-47cc-baa4-852988b5cd5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:17:08.729177 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f6c68f-27ae-47cc-baa4-852988b5cd5e None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:17:08.827352 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 595/1164] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:17:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 07 15:17:08.833325 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490ab27-abd1-4e32-abdb-205ca0ac3770 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:17:08.847760 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:17:08.928007 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490ab27-abd1-4e32-abdb-205ca0ac3770 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:17:08.960508 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e4fce-2768-44e5-bbe4-49fa7f69ad99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:17:08.961290 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e4fce-2768-44e5-bbe4-49fa7f69ad99 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:08.961566 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e4fce-2768-44e5-bbe4-49fa7f69ad99 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:08.961836 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e4fce-2768-44e5-bbe4-49fa7f69ad99 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:08.969028 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25e4fce-2768-44e5-bbe4-49fa7f69ad99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:17:08.970445 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25e4fce-2768-44e5-bbe4-49fa7f69ad99 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:17:08.984371 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-b25e4fce-2768-44e5-bbe4-49fa7f69ad99 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 07 15:17:08.984371 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:17:08.984371 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 07 15:17:08.984371 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 07 15:17:08.984371 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 15:17:08.984371 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 15:17:08.984371 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 15:17:08.984371 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 15:17:08.984371 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 07 15:17:08.984371 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:17:08.984371 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 15:17:08.984371 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:17:08.984371 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:17:08.984371 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:17:08.984371 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:17:08.984371 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:17:08.984371 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:17:08.985579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:17:08.985579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:17:08.985579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:17:08.985579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:17:08.985579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:17:08.985579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:17:08.985579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:17:08.985579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:17:08.985579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:17:08.985579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:17:08.985579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:17:08.985579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:17:08.985579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:17:08.985579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:17:08.985579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:17:08.985579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:17:08.986727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:17:08.986727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:17:08.986727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:17:08.986727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:17:08.986727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:17:08.986727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:17:08.986727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:17:08.986727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:17:08.986727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:17:08.986727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:17:08.986727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 07 15:17:08.986727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:17:08.986727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 07 15:17:08.986727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:17:08.986727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 570/1165] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Jul 7 15:17:08 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:17:08.991297 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6c68f-27ae-47cc-baa4-852988b5cd5e None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:17:09.001781 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:17:09.088562 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6c68f-27ae-47cc-baa4-852988b5cd5e None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:17:09.122095 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf2519-cc10-44e1-b672-012797720750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:17:09.122734 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf2519-cc10-44e1-b672-012797720750 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:09.123015 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf2519-cc10-44e1-b672-012797720750 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:09.123289 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf2519-cc10-44e1-b672-012797720750 None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:09.124036 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf2519-cc10-44e1-b672-012797720750 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:17:09.124337 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf2519-cc10-44e1-b672-012797720750 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:17:09.125687 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf2519-cc10-44e1-b672-012797720750 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:17:09.132561 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 596/1166] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Jul 7 15:17:08 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:17:11.127421 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5e6f2-d353-4f42-8cc9-c18fe932939c None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:11.127969 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5e6f2-d353-4f42-8cc9-c18fe932939c None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:11.127969 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5e6f2-d353-4f42-8cc9-c18fe932939c None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:11.129348 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 571/1167] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:17:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:17:11.134526 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e9f2b-62eb-4a03-bb20-6c416793c021 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:11.134943 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e9f2b-62eb-4a03-bb20-6c416793c021 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:11.135078 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e9f2b-62eb-4a03-bb20-6c416793c021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:11.152367 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-e57e9f2b-62eb-4a03-bb20-6c416793c021 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:17:11.153537 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-e57e9f2b-62eb-4a03-bb20-6c416793c021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:17:11.217171 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 597/1168] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:17:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:17:11.239277 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5e6f2-d353-4f42-8cc9-c18fe932939c None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:17:11.308517 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:17:11.337590 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5e6f2-d353-4f42-8cc9-c18fe932939c None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:17:11.370814 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9bcada-7aa9-4214-92df-9798240a14b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:17:11.371453 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bcada-7aa9-4214-92df-9798240a14b4 None admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:11.371761 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bcada-7aa9-4214-92df-9798240a14b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:11.372036 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bcada-7aa9-4214-92df-9798240a14b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:11.372764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9bcada-7aa9-4214-92df-9798240a14b4 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:17:11.374106 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9bcada-7aa9-4214-92df-9798240a14b4 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:17:11.462353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 572/1169] 10.222.0.25 () {60 vars in 1138 bytes} [Thu Jul 7 15:17:11 2022] POST /identity/v3/domains => generated 267 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:17:13.473225 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509c438-4de1-4cb8-8bfc-6e8112fca8af None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:13.473753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509c438-4de1-4cb8-8bfc-6e8112fca8af None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:13.473753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509c438-4de1-4cb8-8bfc-6e8112fca8af None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:13.475106 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 598/1170] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:17:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:17:13.480261 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422826d-8a1f-4f12-b248-f2a52b7f9c7e None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:13.480690 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422826d-8a1f-4f12-b248-f2a52b7f9c7e None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:13.480827 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422826d-8a1f-4f12-b248-f2a52b7f9c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:13.497711 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-3422826d-8a1f-4f12-b248-f2a52b7f9c7e None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:17:13.498895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-3422826d-8a1f-4f12-b248-f2a52b7f9c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:17:13.564627 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 573/1171] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:17:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:17:13.588732 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509c438-4de1-4cb8-8bfc-6e8112fca8af None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:17:13.591666 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:17:13.686721 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509c438-4de1-4cb8-8bfc-6e8112fca8af None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:17:13.719059 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d750f6a-f6aa-4614-9e1e-e410f4aa58b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:17:13.719919 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d750f6a-f6aa-4614-9e1e-e410f4aa58b5 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:13.720194 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d750f6a-f6aa-4614-9e1e-e410f4aa58b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:13.720461 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d750f6a-f6aa-4614-9e1e-e410f4aa58b5 None admin] PATH_INFO: `/v3/domains/26a2d435902f414b809c793731b6e47f` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:13.732415 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d750f6a-f6aa-4614-9e1e-e410f4aa58b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=26a2d435902f414b809c793731b6e47f)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:17:13.734533 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d750f6a-f6aa-4614-9e1e-e410f4aa58b5 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:17:13.736178 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 599/1172] 10.222.0.25 () {56 vars in 1183 bytes} [Thu Jul 7 15:17:13 2022] GET /identity/v3/domains/26a2d435902f414b809c793731b6e47f => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:17:13.742779 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3422826d-8a1f-4f12-b248-f2a52b7f9c7e None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:17:13.802701 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:17:13.840437 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3422826d-8a1f-4f12-b248-f2a52b7f9c7e None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:17:13.873711 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13a04b-cdbf-4d70-87db-961ab14686c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:17:13.874360 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13a04b-cdbf-4d70-87db-961ab14686c6 None admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:13.874645 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13a04b-cdbf-4d70-87db-961ab14686c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:13.874921 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13a04b-cdbf-4d70-87db-961ab14686c6 None admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:13.875657 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee13a04b-cdbf-4d70-87db-961ab14686c6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:17:13.877081 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee13a04b-cdbf-4d70-87db-961ab14686c6 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:17:13.955954 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 574/1173] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Jul 7 15:17:13 2022] POST /identity/v3/projects => generated 361 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:17:16.003965 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6f5e2-fadd-41b7-bcb4-d303fe4266d2 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:16.004437 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6f5e2-fadd-41b7-bcb4-d303fe4266d2 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:16.004545 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6f5e2-fadd-41b7-bcb4-d303fe4266d2 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:16.005878 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 600/1174] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:17:16.010992 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95b116-d9c8-4b65-b90a-d01bcc17693e None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:16.011439 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95b116-d9c8-4b65-b90a-d01bcc17693e None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:16.011652 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95b116-d9c8-4b65-b90a-d01bcc17693e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:16.028431 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-7b95b116-d9c8-4b65-b90a-d01bcc17693e None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:17:16.029608 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-7b95b116-d9c8-4b65-b90a-d01bcc17693e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:17:16.098925 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 575/1175] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:17:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:17:16.127870 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6f5e2-fadd-41b7-bcb4-d303fe4266d2 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:17:16.188007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:17:16.226172 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6f5e2-fadd-41b7-bcb4-d303fe4266d2 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:17:16.257670 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4023547c-f4b7-44ba-8061-47b96a39575b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:17:16.258512 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023547c-f4b7-44ba-8061-47b96a39575b None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:16.258792 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023547c-f4b7-44ba-8061-47b96a39575b None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:16.259062 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023547c-f4b7-44ba-8061-47b96a39575b None admin] PATH_INFO: `/v3/domains/26a2d435902f414b809c793731b6e47f` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:16.260788 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4023547c-f4b7-44ba-8061-47b96a39575b None admin] RBAC: Authorizing `identity:get_domain(domain_id=26a2d435902f414b809c793731b6e47f)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:17:16.262201 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4023547c-f4b7-44ba-8061-47b96a39575b None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:17:16.263805 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 601/1176] 10.222.0.25 () {56 vars in 1183 bytes} [Thu Jul 7 15:17:16 2022] GET /identity/v3/domains/26a2d435902f414b809c793731b6e47f => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:17:16.269499 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95b116-d9c8-4b65-b90a-d01bcc17693e None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:17:16.312856 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:17:16.370631 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95b116-d9c8-4b65-b90a-d01bcc17693e None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:17:16.412725 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f25762-8eac-49a2-8259-eb5dbd740636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:17:16.413396 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f25762-8eac-49a2-8259-eb5dbd740636 None admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:16.413679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f25762-8eac-49a2-8259-eb5dbd740636 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:16.413956 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f25762-8eac-49a2-8259-eb5dbd740636 None admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:16.414675 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f25762-8eac-49a2-8259-eb5dbd740636 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:17:16.416162 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f25762-8eac-49a2-8259-eb5dbd740636 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:17:16.497718 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 576/1177] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Jul 7 15:17:16 2022] POST /identity/v3/users => generated 311 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:17:18.537064 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36424310-1fd7-4908-b040-cfe757e076e7 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:18.537623 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36424310-1fd7-4908-b040-cfe757e076e7 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:18.537623 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36424310-1fd7-4908-b040-cfe757e076e7 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:18.538953 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 602/1178] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:17:18.544037 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90a3c3-7595-4ed4-9868-ff79be4650d0 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:18.544414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90a3c3-7595-4ed4-9868-ff79be4650d0 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:18.544590 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90a3c3-7595-4ed4-9868-ff79be4650d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:18.560863 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-4b90a3c3-7595-4ed4-9868-ff79be4650d0 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:17:18.562023 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-4b90a3c3-7595-4ed4-9868-ff79be4650d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:17:18.626279 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 577/1179] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:17:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:17:18.649945 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36424310-1fd7-4908-b040-cfe757e076e7 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:17:18.748476 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36424310-1fd7-4908-b040-cfe757e076e7 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:17:18.775477 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:17:18.780692 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b3e71-3e16-434a-9857-5c2b036b3ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:17:18.781464 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b3e71-3e16-434a-9857-5c2b036b3ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:18.781745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b3e71-3e16-434a-9857-5c2b036b3ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:18.782020 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b3e71-3e16-434a-9857-5c2b036b3ea0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:18.792213 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b3e71-3e16-434a-9857-5c2b036b3ea0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:17:18.799678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-db5b3e71-3e16-434a-9857-5c2b036b3ea0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:17:18.799678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:17:18.799678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:17:18.799678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:17:18.799678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:17:18.799678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:17:18.799678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:17:18.799678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:17:18.799678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:17:18.799678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:17:18.799678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:17:18.799678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:17:18.799678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:17:18.799678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:17:18.799678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:17:18.799678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:17:18.800947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:17:18.800947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:17:18.800947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:17:18.800947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:17:18.800947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:17:18.800947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:17:18.800947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:17:18.800947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:17:18.800947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:17:18.800947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:17:18.800947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:17:18.800947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:17:18.800947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:17:18.800947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:17:18.800947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:17:18.800947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:17:18.802066 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:17:18.802066 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:17:18.802066 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:17:18.802066 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:17:18.802066 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:17:18.802066 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:17:18.802066 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:17:18.802066 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:17:18.802066 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:17:18.802066 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:17:18.802066 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:17:18.802066 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:17:18.802066 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 603/1180] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Jul 7 15:17:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:17:18.807173 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90a3c3-7595-4ed4-9868-ff79be4650d0 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:17:18.836184 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:17:18.903536 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90a3c3-7595-4ed4-9868-ff79be4650d0 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:17:18.936233 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43ccbe-44da-46e0-bf76-1d4f8e336f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:17:18.936869 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43ccbe-44da-46e0-bf76-1d4f8e336f5b None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:18.937149 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43ccbe-44da-46e0-bf76-1d4f8e336f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:18.937421 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43ccbe-44da-46e0-bf76-1d4f8e336f5b None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:18.938179 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce43ccbe-44da-46e0-bf76-1d4f8e336f5b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:17:18.938486 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce43ccbe-44da-46e0-bf76-1d4f8e336f5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:17:18.939857 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce43ccbe-44da-46e0-bf76-1d4f8e336f5b None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:17:18.948217 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 578/1181] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:17:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:17:18.953984 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b3e71-3e16-434a-9857-5c2b036b3ea0 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:17:19.050792 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b3e71-3e16-434a-9857-5c2b036b3ea0 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:17:19.084118 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e898d26-2315-48e4-8948-44ff1f8e72be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:17:19.084972 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e898d26-2315-48e4-8948-44ff1f8e72be None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:19.085253 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e898d26-2315-48e4-8948-44ff1f8e72be None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:19.085528 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e898d26-2315-48e4-8948-44ff1f8e72be None admin] PATH_INFO: `/v3/users/41d10d4174d24c5ab965b677e054775d` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:19.099303 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e898d26-2315-48e4-8948-44ff1f8e72be None admin] RBAC: Authorizing `identity:get_user(user_id=41d10d4174d24c5ab965b677e054775d)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:17:19.100763 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e898d26-2315-48e4-8948-44ff1f8e72be None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:17:19.102366 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 604/1182] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:17:18 2022] GET /identity/v3/users/41d10d4174d24c5ab965b677e054775d => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:17:19.108625 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43ccbe-44da-46e0-bf76-1d4f8e336f5b None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:17:19.196123 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:17:19.204332 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43ccbe-44da-46e0-bf76-1d4f8e336f5b None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:17:19.237216 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bbe852-82da-4660-8325-581d36bbaa89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:17:19.237841 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbe852-82da-4660-8325-581d36bbaa89 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:19.238119 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbe852-82da-4660-8325-581d36bbaa89 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:19.238389 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbe852-82da-4660-8325-581d36bbaa89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:19.302161 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bbe852-82da-4660-8325-581d36bbaa89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:17:19.303531 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bbe852-82da-4660-8325-581d36bbaa89 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:17:19.388016 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:17:19.397005 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 579/1183] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:17:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 07 15:17:19.402780 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e898d26-2315-48e4-8948-44ff1f8e72be None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:17:19.450499 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:17:19.499345 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e898d26-2315-48e4-8948-44ff1f8e72be None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:17:19.531675 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96c15c-8325-4c7b-8212-1835ad860c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:17:19.532517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96c15c-8325-4c7b-8212-1835ad860c79 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:19.532799 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96c15c-8325-4c7b-8212-1835ad860c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:19.533072 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96c15c-8325-4c7b-8212-1835ad860c79 None admin] PATH_INFO: `/v3/projects/77d29c9e830c425cbde69025dd3bc8e7` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:19.534245 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96c15c-8325-4c7b-8212-1835ad860c79 None admin] RBAC: Authorizing `identity:get_project(project_id=77d29c9e830c425cbde69025dd3bc8e7)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:17:19.535701 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96c15c-8325-4c7b-8212-1835ad860c79 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:17:19.537343 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 605/1184] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:17:19 2022] GET /identity/v3/projects/77d29c9e830c425cbde69025dd3bc8e7 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:17:19.543896 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bbe852-82da-4660-8325-581d36bbaa89 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:17:19.639445 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bbe852-82da-4660-8325-581d36bbaa89 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:17:19.673942 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8cbcb-3077-4c0e-9a99-cee2dcdc0d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:17:19.674576 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8cbcb-3077-4c0e-9a99-cee2dcdc0d53 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:19.674856 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8cbcb-3077-4c0e-9a99-cee2dcdc0d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:19.675126 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8cbcb-3077-4c0e-9a99-cee2dcdc0d53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:19.675968 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8cbcb-3077-4c0e-9a99-cee2dcdc0d53 None admin] RBAC: Adding query filter params (role.id=0e7793fbf3064037984da651879e3a94, scope.project.id=77d29c9e830c425cbde69025dd3bc8e7, user.id=41d10d4174d24c5ab965b677e054775d) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:17:19.676277 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8cbcb-3077-4c0e-9a99-cee2dcdc0d53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:17:19.677676 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8cbcb-3077-4c0e-9a99-cee2dcdc0d53 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:17:19.682931 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 580/1185] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:17:19 2022] GET /identity/v3/role_assignments?user.id=41d10d4174d24c5ab965b677e054775d&scope.project.id=77d29c9e830c425cbde69025dd3bc8e7&role.id=0e7793fbf3064037984da651879e3a94 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:17:21.634768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c7d13-23ac-4015-a5f6-d81fe5088478 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:21.635318 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c7d13-23ac-4015-a5f6-d81fe5088478 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:21.635318 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c7d13-23ac-4015-a5f6-d81fe5088478 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:21.636678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 606/1186] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:17:21.641905 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e6767-69cc-48fd-8272-b385645338c5 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:21.642316 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e6767-69cc-48fd-8272-b385645338c5 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:21.642464 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e6767-69cc-48fd-8272-b385645338c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:21.658841 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-035e6767-69cc-48fd-8272-b385645338c5 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:17:21.660062 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-035e6767-69cc-48fd-8272-b385645338c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:17:21.708977 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:17:21.725699 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 581/1187] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:17:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:17:21.747785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c7d13-23ac-4015-a5f6-d81fe5088478 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:17:21.843982 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c7d13-23ac-4015-a5f6-d81fe5088478 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:17:21.860219 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:17:21.876751 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a979e7-f775-4684-a58c-a3bff6c53a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:17:21.877513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a979e7-f775-4684-a58c-a3bff6c53a19 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:21.877791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a979e7-f775-4684-a58c-a3bff6c53a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:21.878059 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a979e7-f775-4684-a58c-a3bff6c53a19 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:21.888397 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a979e7-f775-4684-a58c-a3bff6c53a19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:17:21.895828 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-03a979e7-f775-4684-a58c-a3bff6c53a19 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:17:21.895828 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:17:21.895828 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:17:21.895828 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:17:21.895828 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:17:21.895828 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:17:21.895828 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:17:21.895828 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:17:21.895828 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:17:21.895828 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:17:21.895828 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:17:21.895828 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:17:21.895828 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:17:21.895828 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:17:21.895828 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:17:21.895828 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:17:21.896897 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:17:21.896897 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:17:21.896897 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:17:21.896897 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:17:21.896897 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:17:21.896897 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:17:21.896897 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:17:21.896897 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:17:21.896897 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:17:21.896897 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:17:21.896897 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:17:21.896897 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:17:21.896897 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:17:21.896897 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:17:21.896897 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:17:21.896897 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:17:21.898069 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:17:21.898069 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:17:21.898069 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:17:21.898069 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:17:21.898069 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:17:21.898069 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:17:21.898069 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:17:21.898069 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:17:21.898069 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:17:21.898069 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:17:21.898069 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:17:21.898069 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:17:21.898069 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 607/1188] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Jul 7 15:17:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:17:21.903429 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e6767-69cc-48fd-8272-b385645338c5 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:17:21.998186 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e6767-69cc-48fd-8272-b385645338c5 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:17:22.030518 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9317d284-373a-4ab5-bc27-1f8da8cbcdc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:17:22.031162 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317d284-373a-4ab5-bc27-1f8da8cbcdc1 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:22.031534 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317d284-373a-4ab5-bc27-1f8da8cbcdc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:22.031844 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317d284-373a-4ab5-bc27-1f8da8cbcdc1 None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:22.032581 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9317d284-373a-4ab5-bc27-1f8da8cbcdc1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:17:22.032883 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9317d284-373a-4ab5-bc27-1f8da8cbcdc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:17:22.034228 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9317d284-373a-4ab5-bc27-1f8da8cbcdc1 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:17:22.042469 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 582/1189] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:17:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:17:22.048122 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a979e7-f775-4684-a58c-a3bff6c53a19 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:17:22.143668 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a979e7-f775-4684-a58c-a3bff6c53a19 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:17:22.175473 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271af2ee-1ba7-41b8-870b-01fd30bb799f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:17:22.176344 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271af2ee-1ba7-41b8-870b-01fd30bb799f None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:22.176621 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271af2ee-1ba7-41b8-870b-01fd30bb799f None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:22.176889 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271af2ee-1ba7-41b8-870b-01fd30bb799f None admin] PATH_INFO: `/v3/users/41d10d4174d24c5ab965b677e054775d` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:22.178040 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271af2ee-1ba7-41b8-870b-01fd30bb799f None admin] RBAC: Authorizing `identity:get_user(user_id=41d10d4174d24c5ab965b677e054775d)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:17:22.179455 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271af2ee-1ba7-41b8-870b-01fd30bb799f None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:17:22.181047 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 608/1190] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:17:22 2022] GET /identity/v3/users/41d10d4174d24c5ab965b677e054775d => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:17:22.186684 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9317d284-373a-4ab5-bc27-1f8da8cbcdc1 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:17:22.235264 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:17:22.284881 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9317d284-373a-4ab5-bc27-1f8da8cbcdc1 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:17:22.316086 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab9c559-7429-4a0f-9b4f-580d49976c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:17:22.316922 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9c559-7429-4a0f-9b4f-580d49976c5d None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:22.317200 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9c559-7429-4a0f-9b4f-580d49976c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:22.317467 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9c559-7429-4a0f-9b4f-580d49976c5d None admin] PATH_INFO: `/v3/projects/77d29c9e830c425cbde69025dd3bc8e7` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:22.318618 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab9c559-7429-4a0f-9b4f-580d49976c5d None admin] RBAC: Authorizing `identity:get_project(project_id=77d29c9e830c425cbde69025dd3bc8e7)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:17:22.320065 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab9c559-7429-4a0f-9b4f-580d49976c5d None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:17:22.321718 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 583/1191] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:17:22 2022] GET /identity/v3/projects/77d29c9e830c425cbde69025dd3bc8e7 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:17:22.328023 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271af2ee-1ba7-41b8-870b-01fd30bb799f None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:17:22.372062 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:17:22.425507 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271af2ee-1ba7-41b8-870b-01fd30bb799f None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:17:22.457295 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afddfcb-8e67-4050-ab84-89c5decab59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:17:22.458164 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afddfcb-8e67-4050-ab84-89c5decab59d None admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:22.458442 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afddfcb-8e67-4050-ab84-89c5decab59d None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:22.458715 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afddfcb-8e67-4050-ab84-89c5decab59d None admin] PATH_INFO: `/v3/projects/77d29c9e830c425cbde69025dd3bc8e7/users/41d10d4174d24c5ab965b677e054775d/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:22.460653 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afddfcb-8e67-4050-ab84-89c5decab59d None admin] RBAC: Authorizing `identity:create_grant(project_id=77d29c9e830c425cbde69025dd3bc8e7, user_id=41d10d4174d24c5ab965b677e054775d, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:17:22.462071 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afddfcb-8e67-4050-ab84-89c5decab59d None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:17:22.519648 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 609/1192] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Jul 7 15:17:22 2022] PUT /identity/v3/projects/77d29c9e830c425cbde69025dd3bc8e7/users/41d10d4174d24c5ab965b677e054775d/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:17:24.500252 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8be0af-f56b-4a3c-8e0b-8a27c49a728f None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:24.500810 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8be0af-f56b-4a3c-8e0b-8a27c49a728f None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:24.500810 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8be0af-f56b-4a3c-8e0b-8a27c49a728f None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:24.502148 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 584/1193] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:17:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:17:24.507145 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df95349-bc74-4240-bc02-06b413a97f86 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:24.507549 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df95349-bc74-4240-bc02-06b413a97f86 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:24.507792 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df95349-bc74-4240-bc02-06b413a97f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:24.524652 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-8df95349-bc74-4240-bc02-06b413a97f86 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:17:24.526433 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-8df95349-bc74-4240-bc02-06b413a97f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:17:24.590133 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 610/1194] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:17:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:17:24.612722 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8be0af-f56b-4a3c-8e0b-8a27c49a728f None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:17:24.710185 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8be0af-f56b-4a3c-8e0b-8a27c49a728f None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:17:24.726530 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:17:24.743140 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4924b62-1842-4d4b-a01b-7f36b2af4575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:17:24.744453 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4924b62-1842-4d4b-a01b-7f36b2af4575 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:24.749217 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4924b62-1842-4d4b-a01b-7f36b2af4575 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:24.749217 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4924b62-1842-4d4b-a01b-7f36b2af4575 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:24.754657 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4924b62-1842-4d4b-a01b-7f36b2af4575 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:17:24.762836 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-e4924b62-1842-4d4b-a01b-7f36b2af4575 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:17:24.762836 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:17:24.762836 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:17:24.762836 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:17:24.762836 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:17:24.762836 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:17:24.762836 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:17:24.762836 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:17:24.762836 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:17:24.762836 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:17:24.762836 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:17:24.762836 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:17:24.762836 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:17:24.762836 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:17:24.762836 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:17:24.762836 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:17:24.764006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:17:24.764006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:17:24.764006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:17:24.764006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:17:24.764006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:17:24.764006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:17:24.764006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:17:24.764006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:17:24.764006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:17:24.764006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:17:24.764006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:17:24.764006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:17:24.764006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:17:24.764006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:17:24.764006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:17:24.764006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:17:24.765053 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:17:24.765053 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:17:24.765053 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:17:24.765053 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:17:24.765053 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:17:24.765053 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:17:24.765053 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:17:24.765053 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:17:24.765053 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:17:24.765053 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:17:24.765053 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:17:24.765053 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:17:24.765053 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 585/1195] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Jul 7 15:17:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:17:24.769851 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df95349-bc74-4240-bc02-06b413a97f86 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:17:24.801730 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:17:24.865644 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df95349-bc74-4240-bc02-06b413a97f86 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:17:24.899443 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdedaa3-e579-4905-b4e0-56c9b2612571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:17:24.900113 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdedaa3-e579-4905-b4e0-56c9b2612571 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:24.900391 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdedaa3-e579-4905-b4e0-56c9b2612571 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:24.900659 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdedaa3-e579-4905-b4e0-56c9b2612571 None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:24.901391 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdedaa3-e579-4905-b4e0-56c9b2612571 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:17:24.901727 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdedaa3-e579-4905-b4e0-56c9b2612571 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:17:24.903068 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdedaa3-e579-4905-b4e0-56c9b2612571 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:17:24.910789 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 611/1196] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:17:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:17:24.916436 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4924b62-1842-4d4b-a01b-7f36b2af4575 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:17:25.015481 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4924b62-1842-4d4b-a01b-7f36b2af4575 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:17:25.047500 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40861e2b-3b63-46eb-906b-ce9f3742018f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:17:25.048390 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40861e2b-3b63-46eb-906b-ce9f3742018f None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:25.048678 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40861e2b-3b63-46eb-906b-ce9f3742018f None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:25.048956 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40861e2b-3b63-46eb-906b-ce9f3742018f None admin] PATH_INFO: `/v3/users/41d10d4174d24c5ab965b677e054775d` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:25.050142 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40861e2b-3b63-46eb-906b-ce9f3742018f None admin] RBAC: Authorizing `identity:get_user(user_id=41d10d4174d24c5ab965b677e054775d)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:17:25.051556 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40861e2b-3b63-46eb-906b-ce9f3742018f None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:17:25.053206 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 586/1197] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Jul 7 15:17:24 2022] GET /identity/v3/users/41d10d4174d24c5ab965b677e054775d => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:17:25.059358 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdedaa3-e579-4905-b4e0-56c9b2612571 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:17:25.153305 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:17:25.158809 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdedaa3-e579-4905-b4e0-56c9b2612571 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:17:25.190244 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263f6fd-eb49-4ab7-af80-83015ad08be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:17:25.190874 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263f6fd-eb49-4ab7-af80-83015ad08be2 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:25.191153 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263f6fd-eb49-4ab7-af80-83015ad08be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:25.191424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263f6fd-eb49-4ab7-af80-83015ad08be2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:25.256216 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f263f6fd-eb49-4ab7-af80-83015ad08be2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:17:25.257659 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f263f6fd-eb49-4ab7-af80-83015ad08be2 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:17:25.345908 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:17:25.352409 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 612/1198] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:17:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 07 15:17:25.358204 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40861e2b-3b63-46eb-906b-ce9f3742018f None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:17:25.405920 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:17:25.457773 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40861e2b-3b63-46eb-906b-ce9f3742018f None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:17:25.490580 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec8c455-9c86-462a-aae2-64e481172a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:17:25.491427 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8c455-9c86-462a-aae2-64e481172a0e None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:25.491734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8c455-9c86-462a-aae2-64e481172a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:25.492005 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8c455-9c86-462a-aae2-64e481172a0e None admin] PATH_INFO: `/v3/projects/77d29c9e830c425cbde69025dd3bc8e7` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:25.493151 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec8c455-9c86-462a-aae2-64e481172a0e None admin] RBAC: Authorizing `identity:get_project(project_id=77d29c9e830c425cbde69025dd3bc8e7)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:17:25.495056 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec8c455-9c86-462a-aae2-64e481172a0e None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:17:25.496760 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 587/1199] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Jul 7 15:17:25 2022] GET /identity/v3/projects/77d29c9e830c425cbde69025dd3bc8e7 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:17:25.502775 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263f6fd-eb49-4ab7-af80-83015ad08be2 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:17:25.597906 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263f6fd-eb49-4ab7-af80-83015ad08be2 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:17:25.633215 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaef635-1096-43a8-975b-26578d10a3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:17:25.633852 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaef635-1096-43a8-975b-26578d10a3d9 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:17:25.634137 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaef635-1096-43a8-975b-26578d10a3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:17:25.635042 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaef635-1096-43a8-975b-26578d10a3d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:17:25.635900 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaef635-1096-43a8-975b-26578d10a3d9 None admin] RBAC: Adding query filter params (role.id=0e7793fbf3064037984da651879e3a94, scope.project.id=77d29c9e830c425cbde69025dd3bc8e7, user.id=41d10d4174d24c5ab965b677e054775d) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:17:25.636212 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaef635-1096-43a8-975b-26578d10a3d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:17:25.637634 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaef635-1096-43a8-975b-26578d10a3d9 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:17:25.643368 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 613/1200] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:17:25 2022] GET /identity/v3/role_assignments?user.id=41d10d4174d24c5ab965b677e054775d&scope.project.id=77d29c9e830c425cbde69025dd3bc8e7&role.id=0e7793fbf3064037984da651879e3a94 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:17:35.486540 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=44 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:17:35.640750 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=70 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:36.792511 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58cbda5-a27c-4ea0-8aff-092e37f9121a None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:36.793097 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58cbda5-a27c-4ea0-8aff-092e37f9121a None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:36.793097 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58cbda5-a27c-4ea0-8aff-092e37f9121a None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:36.794449 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 588/1201] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:18:36.799626 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7b78e-d458-4c49-833b-0e95e6a2c96b None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:36.800526 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7b78e-d458-4c49-833b-0e95e6a2c96b None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:36.800526 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7b78e-d458-4c49-833b-0e95e6a2c96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:36.817067 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-5aa7b78e-d458-4c49-833b-0e95e6a2c96b None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:18:36.818252 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-5aa7b78e-d458-4c49-833b-0e95e6a2c96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:18:36.883476 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 614/1202] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:18:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:18:36.907300 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58cbda5-a27c-4ea0-8aff-092e37f9121a None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:37.009763 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58cbda5-a27c-4ea0-8aff-092e37f9121a None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:37.042940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b8c89e-7d8d-4d08-88e4-0aae758ae006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:37.043679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8c89e-7d8d-4d08-88e4-0aae758ae006 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:37.044001 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8c89e-7d8d-4d08-88e4-0aae758ae006 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:37.044270 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8c89e-7d8d-4d08-88e4-0aae758ae006 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:37.054144 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b8c89e-7d8d-4d08-88e4-0aae758ae006 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:37.056237 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b8c89e-7d8d-4d08-88e4-0aae758ae006 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:37.073731 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-f3b8c89e-7d8d-4d08-88e4-0aae758ae006 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:18:37.073731 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:18:37.073731 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:18:37.073731 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:18:37.073731 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:18:37.073731 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:18:37.073731 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:18:37.073731 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:18:37.073731 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:18:37.073731 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:18:37.073731 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:18:37.073731 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:18:37.073731 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:18:37.073731 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:18:37.073731 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:18:37.074971 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:18:37.074971 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:18:37.074971 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:18:37.074971 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:18:37.074971 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:18:37.074971 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:18:37.074971 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:18:37.074971 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:18:37.074971 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:18:37.074971 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:18:37.074971 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:18:37.074971 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:18:37.074971 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:18:37.074971 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:18:37.074971 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:18:37.074971 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:18:37.074971 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:18:37.076171 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:18:37.076171 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:18:37.076171 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:18:37.076171 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:18:37.076171 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 589/1203] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:18:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:18:37.080995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7b78e-d458-4c49-833b-0e95e6a2c96b None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:37.177723 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7b78e-d458-4c49-833b-0e95e6a2c96b None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:37.205507 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:37.211067 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce83fe3-a160-4dab-9d9e-d067f9f97bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:37.211770 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce83fe3-a160-4dab-9d9e-d067f9f97bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:37.212054 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce83fe3-a160-4dab-9d9e-d067f9f97bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:37.212329 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce83fe3-a160-4dab-9d9e-d067f9f97bc4 None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:37.213051 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce83fe3-a160-4dab-9d9e-d067f9f97bc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:18:37.213356 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce83fe3-a160-4dab-9d9e-d067f9f97bc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:37.214711 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce83fe3-a160-4dab-9d9e-d067f9f97bc4 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:37.225663 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 615/1204] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:18:37.231882 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b8c89e-7d8d-4d08-88e4-0aae758ae006 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:37.283920 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:37.329202 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b8c89e-7d8d-4d08-88e4-0aae758ae006 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:37.362325 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad1d42-ac35-4ce9-a325-fef74be59f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:37.362973 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad1d42-ac35-4ce9-a325-fef74be59f70 None admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:37.363253 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad1d42-ac35-4ce9-a325-fef74be59f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:37.363526 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad1d42-ac35-4ce9-a325-fef74be59f70 None admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:37.364378 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ad1d42-ac35-4ce9-a325-fef74be59f70 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:37.365790 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ad1d42-ac35-4ce9-a325-fef74be59f70 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:37.447406 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 590/1205] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Jul 7 15:18:37 2022] POST /identity/v3/users => generated 251 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:18:39.469222 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244efaa0-1d27-4c59-b083-9d41a579ee94 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:39.469807 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244efaa0-1d27-4c59-b083-9d41a579ee94 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:39.469807 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244efaa0-1d27-4c59-b083-9d41a579ee94 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:39.471121 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 616/1206] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:18:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:18:39.476957 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153452fd-2b56-4a08-a0cc-dfa9aaa0bba9 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:39.477347 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153452fd-2b56-4a08-a0cc-dfa9aaa0bba9 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:39.477559 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153452fd-2b56-4a08-a0cc-dfa9aaa0bba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:39.495142 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-153452fd-2b56-4a08-a0cc-dfa9aaa0bba9 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:18:39.496362 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-153452fd-2b56-4a08-a0cc-dfa9aaa0bba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:18:39.561992 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 591/1207] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:18:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:18:39.585225 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244efaa0-1d27-4c59-b083-9d41a579ee94 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:39.685413 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244efaa0-1d27-4c59-b083-9d41a579ee94 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:39.718639 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04786729-7608-4e93-b786-b171f3811c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:39.719484 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04786729-7608-4e93-b786-b171f3811c89 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:39.719737 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04786729-7608-4e93-b786-b171f3811c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:39.720012 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04786729-7608-4e93-b786-b171f3811c89 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:39.731324 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04786729-7608-4e93-b786-b171f3811c89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:18:39.738994 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-04786729-7608-4e93-b786-b171f3811c89 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:18:39.738994 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:18:39.738994 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:18:39.738994 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:18:39.738994 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:18:39.738994 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:18:39.738994 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:18:39.738994 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:18:39.738994 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:18:39.738994 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:18:39.738994 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:18:39.738994 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:18:39.738994 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:18:39.738994 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:18:39.738994 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:18:39.738994 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:18:39.740165 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:18:39.740165 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:18:39.740165 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:18:39.740165 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:18:39.740165 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:18:39.740165 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:18:39.740165 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:18:39.740165 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:18:39.740165 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:18:39.740165 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:18:39.740165 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:18:39.740165 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:18:39.740165 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:18:39.740165 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:18:39.740165 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:18:39.740165 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:18:39.741310 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:18:39.741310 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:18:39.741310 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:18:39.741310 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:18:39.741310 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:18:39.741310 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:18:39.741310 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:18:39.741310 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:18:39.741310 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:18:39.741310 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:18:39.741310 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:18:39.741310 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:18:39.741310 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 617/1208] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Jul 7 15:18:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:18:39.746067 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153452fd-2b56-4a08-a0cc-dfa9aaa0bba9 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:39.785010 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:39.842857 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153452fd-2b56-4a08-a0cc-dfa9aaa0bba9 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:39.877525 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bafeef-1442-48ae-9086-97ca8ece0f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:39.878180 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bafeef-1442-48ae-9086-97ca8ece0f1f None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:39.878460 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bafeef-1442-48ae-9086-97ca8ece0f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:39.878729 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bafeef-1442-48ae-9086-97ca8ece0f1f None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:39.879475 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bafeef-1442-48ae-9086-97ca8ece0f1f None admin] RBAC: Adding query filter params (name=service) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:18:39.879813 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bafeef-1442-48ae-9086-97ca8ece0f1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:39.881188 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bafeef-1442-48ae-9086-97ca8ece0f1f None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:39.889538 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 592/1209] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Jul 7 15:18:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:18:39.895230 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04786729-7608-4e93-b786-b171f3811c89 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:39.944719 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:39.995186 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04786729-7608-4e93-b786-b171f3811c89 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:40.029241 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62313f2c-6ec8-4b0a-b0c9-dfdddb616a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:40.030029 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62313f2c-6ec8-4b0a-b0c9-dfdddb616a30 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:40.030312 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62313f2c-6ec8-4b0a-b0c9-dfdddb616a30 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:40.030590 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62313f2c-6ec8-4b0a-b0c9-dfdddb616a30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:40.041045 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62313f2c-6ec8-4b0a-b0c9-dfdddb616a30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:40.042473 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62313f2c-6ec8-4b0a-b0c9-dfdddb616a30 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:40.060073 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-62313f2c-6ec8-4b0a-b0c9-dfdddb616a30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:18:40.060073 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:18:40.060073 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:18:40.060073 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:18:40.060073 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:18:40.060073 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:18:40.060073 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:18:40.060073 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:18:40.060073 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:18:40.060073 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:18:40.060073 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:18:40.060073 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:18:40.060073 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:18:40.060073 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:18:40.060073 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:18:40.062389 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:18:40.062389 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:18:40.062389 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:18:40.062389 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:18:40.062389 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:18:40.062389 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:18:40.062389 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:18:40.062389 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:18:40.062389 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:18:40.062389 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:18:40.062389 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:18:40.062389 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:18:40.062389 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:18:40.062389 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:18:40.062389 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:18:40.062389 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:18:40.062389 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:18:40.063538 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:18:40.063538 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:18:40.063538 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:18:40.063538 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:18:40.063538 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 618/1210] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:18:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:18:40.067110 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bafeef-1442-48ae-9086-97ca8ece0f1f None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:40.162793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:40.166116 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bafeef-1442-48ae-9086-97ca8ece0f1f None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:40.198589 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b44f49-fa1d-43ba-9e2b-37a169e38ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:40.199238 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b44f49-fa1d-43ba-9e2b-37a169e38ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:40.199518 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b44f49-fa1d-43ba-9e2b-37a169e38ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:40.199819 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b44f49-fa1d-43ba-9e2b-37a169e38ac3 None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:40.200534 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b44f49-fa1d-43ba-9e2b-37a169e38ac3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:18:40.200837 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b44f49-fa1d-43ba-9e2b-37a169e38ac3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:40.202217 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b44f49-fa1d-43ba-9e2b-37a169e38ac3 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:40.213182 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 593/1211] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:18:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:18:40.218951 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62313f2c-6ec8-4b0a-b0c9-dfdddb616a30 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:40.318337 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62313f2c-6ec8-4b0a-b0c9-dfdddb616a30 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:40.319837 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:40.351948 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97407e72-6117-48de-b385-8f2d7901af34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:40.352791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97407e72-6117-48de-b385-8f2d7901af34 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:40.353212 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97407e72-6117-48de-b385-8f2d7901af34 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:40.353860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97407e72-6117-48de-b385-8f2d7901af34 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:40.361475 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97407e72-6117-48de-b385-8f2d7901af34 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:40.362920 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97407e72-6117-48de-b385-8f2d7901af34 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:40.377678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-97407e72-6117-48de-b385-8f2d7901af34 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 07 15:18:40.377678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:18:40.377678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:18:40.377678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:18:40.377678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:18:40.377678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:18:40.377678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:18:40.377678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:18:40.377678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:18:40.377678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:18:40.377678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:18:40.377678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:18:40.377678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 15:18:40.377678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 15:18:40.377678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 15:18:40.377678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 15:18:40.379661 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:18:40.379661 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:18:40.379661 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 15:18:40.379661 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:18:40.379661 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 15:18:40.379661 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:18:40.379661 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:18:40.379661 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:18:40.379661 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:18:40.379661 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:18:40.379661 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:18:40.379661 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:18:40.379661 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:18:40.379661 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:18:40.379661 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:18:40.379661 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:18:40.379661 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:18:40.379661 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:18:40.380900 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:18:40.380900 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:18:40.380900 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 15:18:40.380900 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 15:18:40.380900 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 15:18:40.380900 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 15:18:40.380900 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 15:18:40.380900 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 15:18:40.380900 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 07 15:18:40.380900 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:18:40.380900 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 619/1212] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:18:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:18:40.384630 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b44f49-fa1d-43ba-9e2b-37a169e38ac3 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:40.486665 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b44f49-fa1d-43ba-9e2b-37a169e38ac3 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:40.523649 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558b70d5-c5c9-480f-b47c-91ca75c946e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:40.524326 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b70d5-c5c9-480f-b47c-91ca75c946e0 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:40.524609 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b70d5-c5c9-480f-b47c-91ca75c946e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:40.524885 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b70d5-c5c9-480f-b47c-91ca75c946e0 None admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:40.525719 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558b70d5-c5c9-480f-b47c-91ca75c946e0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:18:40.526028 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558b70d5-c5c9-480f-b47c-91ca75c946e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:40.527585 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558b70d5-c5c9-480f-b47c-91ca75c946e0 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:40.545516 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 594/1213] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Jul 7 15:18:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:18:40.551240 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97407e72-6117-48de-b385-8f2d7901af34 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:40.649845 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97407e72-6117-48de-b385-8f2d7901af34 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:40.683230 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce85adf-2aa7-4c7e-b07d-2fde56a8b746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:40.684301 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce85adf-2aa7-4c7e-b07d-2fde56a8b746 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:40.684772 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce85adf-2aa7-4c7e-b07d-2fde56a8b746 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:40.685256 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce85adf-2aa7-4c7e-b07d-2fde56a8b746 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:40.716529 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:40.753817 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce85adf-2aa7-4c7e-b07d-2fde56a8b746 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:40.755271 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce85adf-2aa7-4c7e-b07d-2fde56a8b746 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:40.856039 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 620/1214] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:18:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 07 15:18:40.861951 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558b70d5-c5c9-480f-b47c-91ca75c946e0 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:40.880940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:40.960415 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558b70d5-c5c9-480f-b47c-91ca75c946e0 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:40.996442 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555e105-e92c-40c2-8a6c-3d6b8beb11ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:40.997224 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555e105-e92c-40c2-8a6c-3d6b8beb11ba None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:40.997501 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555e105-e92c-40c2-8a6c-3d6b8beb11ba None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:40.997833 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555e105-e92c-40c2-8a6c-3d6b8beb11ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:41.008340 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6555e105-e92c-40c2-8a6c-3d6b8beb11ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:41.009745 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6555e105-e92c-40c2-8a6c-3d6b8beb11ba None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:41.027752 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-6555e105-e92c-40c2-8a6c-3d6b8beb11ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:18:41.027752 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:18:41.027752 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:18:41.027752 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:18:41.027752 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:18:41.027752 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:18:41.027752 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:18:41.027752 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:18:41.027752 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:18:41.027752 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:18:41.027752 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:18:41.027752 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:18:41.027752 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:18:41.027752 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:18:41.027752 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:18:41.028832 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:18:41.028832 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:18:41.028832 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:18:41.028832 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:18:41.028832 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:18:41.028832 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:18:41.028832 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:18:41.028832 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:18:41.028832 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:18:41.028832 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:18:41.028832 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:18:41.028832 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:18:41.028832 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:18:41.028832 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:18:41.028832 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:18:41.028832 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:18:41.028832 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:18:41.029978 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:18:41.029978 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:18:41.029978 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:18:41.029978 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:18:41.029978 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 595/1215] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:18:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:18:41.034704 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce85adf-2aa7-4c7e-b07d-2fde56a8b746 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:41.093858 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:41.135041 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce85adf-2aa7-4c7e-b07d-2fde56a8b746 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:41.168141 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e444c-b74c-4e4b-91b7-18874bfe05a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:41.168798 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e444c-b74c-4e4b-91b7-18874bfe05a4 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:41.169087 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e444c-b74c-4e4b-91b7-18874bfe05a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:41.169421 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e444c-b74c-4e4b-91b7-18874bfe05a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:41.170143 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e444c-b74c-4e4b-91b7-18874bfe05a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:18:41.170450 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e444c-b74c-4e4b-91b7-18874bfe05a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:41.171846 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e444c-b74c-4e4b-91b7-18874bfe05a4 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:41.183179 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 621/1216] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:18:41.188902 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555e105-e92c-40c2-8a6c-3d6b8beb11ba None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:41.258251 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:41.288170 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555e105-e92c-40c2-8a6c-3d6b8beb11ba None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:41.321617 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c7332-81ac-4de0-9540-0a98bd27af44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:41.322428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c7332-81ac-4de0-9540-0a98bd27af44 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:41.322708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c7332-81ac-4de0-9540-0a98bd27af44 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:41.322977 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c7332-81ac-4de0-9540-0a98bd27af44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:41.329521 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055c7332-81ac-4de0-9540-0a98bd27af44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:41.330936 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055c7332-81ac-4de0-9540-0a98bd27af44 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:41.344950 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-055c7332-81ac-4de0-9540-0a98bd27af44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:18:41.344950 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:18:41.344950 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:18:41.344950 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:18:41.344950 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:18:41.344950 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:18:41.344950 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:18:41.344950 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:18:41.344950 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:18:41.344950 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:18:41.344950 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:18:41.344950 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:18:41.344950 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 15:18:41.344950 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 15:18:41.344950 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 15:18:41.346024 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 15:18:41.346024 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:18:41.346024 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:18:41.346024 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:18:41.346024 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:18:41.346024 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:18:41.346024 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:18:41.346024 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:18:41.346024 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:18:41.346024 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:18:41.346024 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:18:41.346024 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:18:41.346024 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:18:41.346024 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:18:41.346024 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:18:41.346024 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:18:41.346024 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:18:41.347372 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 15:18:41.347372 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 15:18:41.347372 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 15:18:41.347372 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 15:18:41.347372 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 15:18:41.347372 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 15:18:41.347372 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:18:41.347372 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:18:41.347372 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 596/1217] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:18:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:18:41.352393 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e444c-b74c-4e4b-91b7-18874bfe05a4 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:41.449187 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e444c-b74c-4e4b-91b7-18874bfe05a4 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:41.470510 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:41.481933 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f611e-f26a-449e-bf0b-21b58f96ea2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:41.482583 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f611e-f26a-449e-bf0b-21b58f96ea2c None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:41.482866 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f611e-f26a-449e-bf0b-21b58f96ea2c None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:41.483142 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f611e-f26a-449e-bf0b-21b58f96ea2c None admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:41.483972 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f611e-f26a-449e-bf0b-21b58f96ea2c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:18:41.484280 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f611e-f26a-449e-bf0b-21b58f96ea2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:41.485698 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f611e-f26a-449e-bf0b-21b58f96ea2c None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:41.497745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 622/1218] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Jul 7 15:18:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:18:41.503824 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c7332-81ac-4de0-9540-0a98bd27af44 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:41.603333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c7332-81ac-4de0-9540-0a98bd27af44 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:41.633429 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:41.639129 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907e6be-7698-445f-9470-a4b7b02ecdb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:41.639807 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907e6be-7698-445f-9470-a4b7b02ecdb9 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:41.640090 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907e6be-7698-445f-9470-a4b7b02ecdb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:41.640362 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907e6be-7698-445f-9470-a4b7b02ecdb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:41.641186 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c907e6be-7698-445f-9470-a4b7b02ecdb9 None admin] RBAC: Adding query filter params (role.id=3d4f4e0a472e4ff9ac68d90c8612e956, scope.project.id=c5260c7fd3614e9e93b167c86a94260e, user.id=755c2782cfb042f89950b8fc862e4fa1) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:18:41.641491 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c907e6be-7698-445f-9470-a4b7b02ecdb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:41.642899 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c907e6be-7698-445f-9470-a4b7b02ecdb9 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:41.648249 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 597/1219] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:18:41 2022] GET /identity/v3/role_assignments?user.id=755c2782cfb042f89950b8fc862e4fa1&scope.project.id=c5260c7fd3614e9e93b167c86a94260e&role.id=3d4f4e0a472e4ff9ac68d90c8612e956 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:18:43.650669 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a98bd8-de44-4f8a-9959-f8d5f7dda58e None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:43.651197 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a98bd8-de44-4f8a-9959-f8d5f7dda58e None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:43.651197 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a98bd8-de44-4f8a-9959-f8d5f7dda58e None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:43.652668 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 623/1220] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:18:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:18:43.658039 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a4384-0956-41f1-82a1-e648d2b167ad None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:43.658466 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a4384-0956-41f1-82a1-e648d2b167ad None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:43.658589 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a4384-0956-41f1-82a1-e648d2b167ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:43.675756 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-215a4384-0956-41f1-82a1-e648d2b167ad None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:18:43.676932 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-215a4384-0956-41f1-82a1-e648d2b167ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:18:43.742175 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 598/1221] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:18:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:18:43.764284 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a98bd8-de44-4f8a-9959-f8d5f7dda58e None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:43.871704 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a98bd8-de44-4f8a-9959-f8d5f7dda58e None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:43.905142 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a732388-32eb-45e9-91e1-6402a43ffb7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:43.905856 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a732388-32eb-45e9-91e1-6402a43ffb7e None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:43.906134 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a732388-32eb-45e9-91e1-6402a43ffb7e None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:43.906404 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a732388-32eb-45e9-91e1-6402a43ffb7e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:43.916330 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a732388-32eb-45e9-91e1-6402a43ffb7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:18:43.924007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-4a732388-32eb-45e9-91e1-6402a43ffb7e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:18:43.924007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:18:43.924007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:18:43.924007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:18:43.924007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:18:43.924007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:18:43.924007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:18:43.924007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:18:43.924007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:18:43.924007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:18:43.924007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:18:43.924007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:18:43.924007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:18:43.924007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:18:43.924007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:18:43.924007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:18:43.928570 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:18:43.928570 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:18:43.928570 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:18:43.928570 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:18:43.928570 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:18:43.928570 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:18:43.928570 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:18:43.928570 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:18:43.928570 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:18:43.928570 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:18:43.928570 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:18:43.928570 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:18:43.928570 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:18:43.928570 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:18:43.928570 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:18:43.928570 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:18:43.929440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:18:43.929440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:18:43.929440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:18:43.929440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:18:43.929440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:18:43.929440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:18:43.929440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:18:43.929440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:18:43.929440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:18:43.929440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:18:43.929440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:18:43.929440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:18:43.929440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 624/1222] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Jul 7 15:18:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:18:43.931314 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a4384-0956-41f1-82a1-e648d2b167ad None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:44.035063 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a4384-0956-41f1-82a1-e648d2b167ad None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:44.058960 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:44.069087 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae22e1-9e10-473b-85bb-2e296ac9bcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:44.069746 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae22e1-9e10-473b-85bb-2e296ac9bcbd None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:44.070580 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae22e1-9e10-473b-85bb-2e296ac9bcbd None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:44.070853 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae22e1-9e10-473b-85bb-2e296ac9bcbd None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:44.071612 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecae22e1-9e10-473b-85bb-2e296ac9bcbd None admin] RBAC: Adding query filter params (name=service) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:18:44.071920 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecae22e1-9e10-473b-85bb-2e296ac9bcbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:44.073264 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecae22e1-9e10-473b-85bb-2e296ac9bcbd None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:44.080971 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 599/1223] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Jul 7 15:18:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:18:44.087383 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a732388-32eb-45e9-91e1-6402a43ffb7e None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:44.129777 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:44.184763 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a732388-32eb-45e9-91e1-6402a43ffb7e None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:44.218332 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376a0c6-1fe1-4fbb-ae2a-c51782eb26f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:44.219113 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376a0c6-1fe1-4fbb-ae2a-c51782eb26f1 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:44.219392 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376a0c6-1fe1-4fbb-ae2a-c51782eb26f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:44.219692 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376a0c6-1fe1-4fbb-ae2a-c51782eb26f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:44.230344 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c376a0c6-1fe1-4fbb-ae2a-c51782eb26f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:44.231798 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c376a0c6-1fe1-4fbb-ae2a-c51782eb26f1 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:44.249094 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-c376a0c6-1fe1-4fbb-ae2a-c51782eb26f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:18:44.249094 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:18:44.249094 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:18:44.249094 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:18:44.249094 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:18:44.249094 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:18:44.249094 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:18:44.249094 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:18:44.249094 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:18:44.249094 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:18:44.249094 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:18:44.249094 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:18:44.249094 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:18:44.249094 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:18:44.249094 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:18:44.249998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:18:44.249998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:18:44.249998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:18:44.249998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:18:44.249998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:18:44.249998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:18:44.249998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:18:44.249998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:18:44.249998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:18:44.249998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:18:44.249998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:18:44.249998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:18:44.249998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:18:44.249998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:18:44.249998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:18:44.249998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:18:44.249998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:18:44.251150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:18:44.251150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:18:44.251150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:18:44.251150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:18:44.251150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 625/1224] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:18:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:18:44.256086 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae22e1-9e10-473b-85bb-2e296ac9bcbd None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:44.354267 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae22e1-9e10-473b-85bb-2e296ac9bcbd None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:44.387858 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ccf4b-ce51-4bd7-8dd1-7458fc6499b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:44.388503 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ccf4b-ce51-4bd7-8dd1-7458fc6499b4 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:44.388780 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ccf4b-ce51-4bd7-8dd1-7458fc6499b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:44.389054 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ccf4b-ce51-4bd7-8dd1-7458fc6499b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:44.389770 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43ccf4b-ce51-4bd7-8dd1-7458fc6499b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:18:44.390075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43ccf4b-ce51-4bd7-8dd1-7458fc6499b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:44.391423 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43ccf4b-ce51-4bd7-8dd1-7458fc6499b4 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:44.403098 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 600/1225] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:18:44.408929 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376a0c6-1fe1-4fbb-ae2a-c51782eb26f1 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:44.505304 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:44.512262 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376a0c6-1fe1-4fbb-ae2a-c51782eb26f1 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:44.545741 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af1fd4-db72-417e-b20e-f38acbded0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:44.546522 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af1fd4-db72-417e-b20e-f38acbded0cf None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:44.546798 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af1fd4-db72-417e-b20e-f38acbded0cf None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:44.547068 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af1fd4-db72-417e-b20e-f38acbded0cf None admin] PATH_INFO: `/v3/users/placement` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:44.554930 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59af1fd4-db72-417e-b20e-f38acbded0cf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:44.556504 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59af1fd4-db72-417e-b20e-f38acbded0cf None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:44.570670 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-59af1fd4-db72-417e-b20e-f38acbded0cf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 07 15:18:44.570670 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:18:44.570670 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:18:44.570670 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:18:44.570670 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:18:44.570670 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:18:44.570670 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:18:44.570670 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:18:44.570670 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:18:44.570670 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:18:44.570670 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:18:44.570670 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:18:44.570670 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 15:18:44.570670 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 15:18:44.570670 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 15:18:44.570670 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 15:18:44.571892 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:18:44.571892 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:18:44.571892 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 15:18:44.571892 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:18:44.571892 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 15:18:44.571892 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:18:44.571892 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:18:44.571892 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:18:44.571892 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:18:44.571892 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:18:44.571892 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:18:44.571892 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:18:44.571892 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:18:44.571892 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:18:44.571892 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:18:44.571892 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:18:44.571892 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:18:44.571892 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:18:44.573093 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:18:44.573093 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:18:44.573093 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 15:18:44.573093 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 15:18:44.573093 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 15:18:44.573093 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 15:18:44.573093 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 15:18:44.573093 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 15:18:44.573093 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 07 15:18:44.573093 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:18:44.573093 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 626/1226] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:18:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:18:44.577649 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ccf4b-ce51-4bd7-8dd1-7458fc6499b4 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:44.623878 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:44.674702 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ccf4b-ce51-4bd7-8dd1-7458fc6499b4 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:44.708505 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437fe136-ddf4-4b53-8f6c-164cfb781f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:44.709159 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fe136-ddf4-4b53-8f6c-164cfb781f7b None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:44.709439 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fe136-ddf4-4b53-8f6c-164cfb781f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:44.709714 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fe136-ddf4-4b53-8f6c-164cfb781f7b None admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:44.711146 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437fe136-ddf4-4b53-8f6c-164cfb781f7b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:18:44.711477 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437fe136-ddf4-4b53-8f6c-164cfb781f7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:44.712918 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437fe136-ddf4-4b53-8f6c-164cfb781f7b None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:44.730391 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 601/1227] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Jul 7 15:18:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:18:44.736017 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af1fd4-db72-417e-b20e-f38acbded0cf None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:44.835361 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af1fd4-db72-417e-b20e-f38acbded0cf None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:44.868567 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76415ff6-cc52-43e9-81fd-e5e8b2636818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:44.869335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76415ff6-cc52-43e9-81fd-e5e8b2636818 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:44.869619 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76415ff6-cc52-43e9-81fd-e5e8b2636818 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:44.869898 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76415ff6-cc52-43e9-81fd-e5e8b2636818 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:44.880404 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76415ff6-cc52-43e9-81fd-e5e8b2636818 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:44.881900 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76415ff6-cc52-43e9-81fd-e5e8b2636818 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:44.900926 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-76415ff6-cc52-43e9-81fd-e5e8b2636818 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:18:44.900926 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:18:44.900926 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:18:44.900926 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:18:44.900926 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:18:44.900926 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:18:44.900926 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:18:44.900926 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:18:44.900926 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:18:44.900926 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:18:44.900926 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:18:44.900926 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:18:44.900926 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:18:44.900926 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:18:44.900926 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:18:44.902029 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:18:44.902029 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:18:44.902029 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:18:44.902029 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:18:44.902029 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:18:44.902029 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:18:44.902029 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:18:44.902029 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:18:44.902029 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:18:44.902029 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:18:44.902029 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:18:44.902029 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:18:44.902029 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:18:44.902029 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:18:44.902029 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:18:44.902029 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:18:44.902029 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:18:44.903300 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:18:44.903300 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:18:44.903300 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:18:44.903300 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:18:44.903300 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 627/1228] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:18:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:18:44.907911 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437fe136-ddf4-4b53-8f6c-164cfb781f7b None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:45.009017 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437fe136-ddf4-4b53-8f6c-164cfb781f7b None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:45.011303 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:45.041745 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d2a2f-98b2-44a2-bef7-0547977fc752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:45.042468 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d2a2f-98b2-44a2-bef7-0547977fc752 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:45.042747 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d2a2f-98b2-44a2-bef7-0547977fc752 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:45.043035 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d2a2f-98b2-44a2-bef7-0547977fc752 None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:45.043776 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35d2a2f-98b2-44a2-bef7-0547977fc752 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:18:45.044081 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35d2a2f-98b2-44a2-bef7-0547977fc752 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:45.045427 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35d2a2f-98b2-44a2-bef7-0547977fc752 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:45.059102 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 602/1229] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:18:45.064651 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76415ff6-cc52-43e9-81fd-e5e8b2636818 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:45.074515 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:45.163065 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76415ff6-cc52-43e9-81fd-e5e8b2636818 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:45.197005 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88128f4-e37c-46bc-97f9-cf10e732dba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:45.197774 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88128f4-e37c-46bc-97f9-cf10e732dba1 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:45.198053 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88128f4-e37c-46bc-97f9-cf10e732dba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:45.198322 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88128f4-e37c-46bc-97f9-cf10e732dba1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:45.204828 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88128f4-e37c-46bc-97f9-cf10e732dba1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:45.206251 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88128f4-e37c-46bc-97f9-cf10e732dba1 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:45.220248 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-f88128f4-e37c-46bc-97f9-cf10e732dba1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:18:45.220248 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:18:45.220248 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:18:45.220248 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:18:45.220248 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:18:45.220248 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:18:45.220248 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:18:45.220248 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:18:45.220248 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:18:45.220248 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:18:45.220248 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:18:45.220248 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:18:45.220248 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 15:18:45.220248 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 15:18:45.220248 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 15:18:45.221560 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 15:18:45.221560 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:18:45.221560 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:18:45.221560 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:18:45.221560 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:18:45.221560 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:18:45.221560 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:18:45.221560 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:18:45.221560 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:18:45.221560 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:18:45.221560 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:18:45.221560 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:18:45.221560 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:18:45.221560 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:18:45.221560 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:18:45.221560 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:18:45.221560 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:18:45.222703 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 15:18:45.222703 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 15:18:45.222703 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 15:18:45.222703 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 15:18:45.222703 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 15:18:45.222703 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 15:18:45.222703 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:18:45.222703 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:18:45.222703 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 628/1230] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:18:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:18:45.227539 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d2a2f-98b2-44a2-bef7-0547977fc752 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:45.325674 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d2a2f-98b2-44a2-bef7-0547977fc752 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:45.358491 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2051b8-a434-4994-99c2-ea3975febf96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:45.359134 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2051b8-a434-4994-99c2-ea3975febf96 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:45.359415 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2051b8-a434-4994-99c2-ea3975febf96 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:45.359729 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2051b8-a434-4994-99c2-ea3975febf96 None admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:45.360505 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2051b8-a434-4994-99c2-ea3975febf96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:18:45.360812 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2051b8-a434-4994-99c2-ea3975febf96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:45.362228 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2051b8-a434-4994-99c2-ea3975febf96 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:45.374861 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 603/1231] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Jul 7 15:18:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:18:45.380743 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88128f4-e37c-46bc-97f9-cf10e732dba1 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:45.443869 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:45.476633 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88128f4-e37c-46bc-97f9-cf10e732dba1 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:45.511168 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93518c8f-b6c2-4c84-942c-b92f84ee7715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:45.512071 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93518c8f-b6c2-4c84-942c-b92f84ee7715 None admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:45.512356 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93518c8f-b6c2-4c84-942c-b92f84ee7715 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:45.512633 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93518c8f-b6c2-4c84-942c-b92f84ee7715 None admin] PATH_INFO: `/v3/projects/c5260c7fd3614e9e93b167c86a94260e/users/755c2782cfb042f89950b8fc862e4fa1/roles/3d4f4e0a472e4ff9ac68d90c8612e956` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:45.526611 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93518c8f-b6c2-4c84-942c-b92f84ee7715 None admin] RBAC: Authorizing `identity:create_grant(project_id=c5260c7fd3614e9e93b167c86a94260e, user_id=755c2782cfb042f89950b8fc862e4fa1, role_id=3d4f4e0a472e4ff9ac68d90c8612e956)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:45.530277 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93518c8f-b6c2-4c84-942c-b92f84ee7715 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:45.597069 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 629/1232] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Jul 7 15:18:45 2022] PUT /identity/v3/projects/c5260c7fd3614e9e93b167c86a94260e/users/755c2782cfb042f89950b8fc862e4fa1/roles/3d4f4e0a472e4ff9ac68d90c8612e956 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:18:47.622144 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835aa6c3-668a-43bd-82ca-4debb74dbed3 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:47.622733 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835aa6c3-668a-43bd-82ca-4debb74dbed3 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:47.622733 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835aa6c3-668a-43bd-82ca-4debb74dbed3 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:47.624143 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 604/1233] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:18:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:18:47.629050 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e37421-7adf-4bc6-9344-5c5b9abda88c None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:47.629442 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e37421-7adf-4bc6-9344-5c5b9abda88c None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:47.629606 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e37421-7adf-4bc6-9344-5c5b9abda88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:47.645613 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-97e37421-7adf-4bc6-9344-5c5b9abda88c None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:18:47.646860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-97e37421-7adf-4bc6-9344-5c5b9abda88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:18:47.712514 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 630/1234] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:18:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:18:47.736290 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835aa6c3-668a-43bd-82ca-4debb74dbed3 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:47.762735 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:47.843484 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835aa6c3-668a-43bd-82ca-4debb74dbed3 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:47.877601 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c444cea-111e-440b-9ce8-cf5ddebfc900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:47.878314 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c444cea-111e-440b-9ce8-cf5ddebfc900 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:47.878591 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c444cea-111e-440b-9ce8-cf5ddebfc900 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:47.878859 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c444cea-111e-440b-9ce8-cf5ddebfc900 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:47.888919 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c444cea-111e-440b-9ce8-cf5ddebfc900 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:18:47.896614 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-3c444cea-111e-440b-9ce8-cf5ddebfc900 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:18:47.896614 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:18:47.896614 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:18:47.896614 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:18:47.896614 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:18:47.896614 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:18:47.896614 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:18:47.896614 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:18:47.896614 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:18:47.896614 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:18:47.896614 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:18:47.896614 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:18:47.896614 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:18:47.896614 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:18:47.896614 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:18:47.896614 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:18:47.897812 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:18:47.897812 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:18:47.897812 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:18:47.897812 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:18:47.897812 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:18:47.897812 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:18:47.897812 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:18:47.897812 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:18:47.897812 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:18:47.897812 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:18:47.897812 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:18:47.897812 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:18:47.897812 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:18:47.897812 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:18:47.897812 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:18:47.897812 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:18:47.899539 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:18:47.899539 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:18:47.899539 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:18:47.899539 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:18:47.899539 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:18:47.899539 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:18:47.899539 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:18:47.899539 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:18:47.899539 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:18:47.899539 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:18:47.899539 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 15:18:47.899539 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:18:47.899539 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 605/1235] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Jul 7 15:18:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:18:47.904383 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e37421-7adf-4bc6-9344-5c5b9abda88c None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:47.949726 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:48.003621 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e37421-7adf-4bc6-9344-5c5b9abda88c None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:48.041053 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa8635-944d-445b-98fa-2596cd20d856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:48.041738 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa8635-944d-445b-98fa-2596cd20d856 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:48.042018 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa8635-944d-445b-98fa-2596cd20d856 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:48.042295 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa8635-944d-445b-98fa-2596cd20d856 None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:48.043036 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa8635-944d-445b-98fa-2596cd20d856 None admin] RBAC: Adding query filter params (name=service) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:18:48.043343 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa8635-944d-445b-98fa-2596cd20d856 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:48.044740 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa8635-944d-445b-98fa-2596cd20d856 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:48.052848 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 631/1236] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Jul 7 15:18:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:18:48.058591 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c444cea-111e-440b-9ce8-cf5ddebfc900 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:48.150125 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:48.167393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c444cea-111e-440b-9ce8-cf5ddebfc900 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:48.201315 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d7321-b20f-4b72-bba5-3de92cac56a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:48.202098 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d7321-b20f-4b72-bba5-3de92cac56a9 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:48.202379 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d7321-b20f-4b72-bba5-3de92cac56a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:48.202651 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d7321-b20f-4b72-bba5-3de92cac56a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:48.213544 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637d7321-b20f-4b72-bba5-3de92cac56a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:48.214986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637d7321-b20f-4b72-bba5-3de92cac56a9 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:48.232986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-637d7321-b20f-4b72-bba5-3de92cac56a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:18:48.232986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:18:48.232986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:18:48.232986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:18:48.232986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:18:48.232986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:18:48.232986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:18:48.232986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:18:48.232986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:18:48.232986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:18:48.232986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:18:48.232986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:18:48.232986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:18:48.232986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:18:48.232986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:18:48.234092 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:18:48.234092 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:18:48.234092 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:18:48.234092 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:18:48.234092 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:18:48.234092 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:18:48.234092 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:18:48.234092 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:18:48.234092 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:18:48.234092 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:18:48.234092 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:18:48.234092 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:18:48.234092 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:18:48.234092 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:18:48.234092 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:18:48.234092 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:18:48.234092 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:18:48.235225 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:18:48.235225 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:18:48.235225 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:18:48.235225 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:18:48.235225 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 606/1237] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:18:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:18:48.240072 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa8635-944d-445b-98fa-2596cd20d856 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:48.337602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa8635-944d-445b-98fa-2596cd20d856 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:48.346858 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:48.370290 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a14da93-bb03-43b6-a9d6-f3b4114cb07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:48.370937 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14da93-bb03-43b6-a9d6-f3b4114cb07f None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:48.371222 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14da93-bb03-43b6-a9d6-f3b4114cb07f None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:48.371559 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14da93-bb03-43b6-a9d6-f3b4114cb07f None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:48.372303 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a14da93-bb03-43b6-a9d6-f3b4114cb07f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:18:48.372606 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a14da93-bb03-43b6-a9d6-f3b4114cb07f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:48.373983 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a14da93-bb03-43b6-a9d6-f3b4114cb07f None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:48.385206 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 632/1238] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:18:48.390783 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d7321-b20f-4b72-bba5-3de92cac56a9 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:48.495872 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d7321-b20f-4b72-bba5-3de92cac56a9 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:48.530279 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea64528-db5f-483b-b060-88b245c9b92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:48.531068 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea64528-db5f-483b-b060-88b245c9b92f None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:48.531347 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea64528-db5f-483b-b060-88b245c9b92f None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:48.531645 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea64528-db5f-483b-b060-88b245c9b92f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:48.534302 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:48.538758 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea64528-db5f-483b-b060-88b245c9b92f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:48.540234 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea64528-db5f-483b-b060-88b245c9b92f None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:48.554409 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-bea64528-db5f-483b-b060-88b245c9b92f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 07 15:18:48.554409 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:18:48.554409 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:18:48.554409 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:18:48.554409 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:18:48.554409 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:18:48.554409 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:18:48.554409 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:18:48.554409 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:18:48.554409 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:18:48.554409 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:18:48.554409 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:18:48.554409 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 15:18:48.554409 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 15:18:48.554409 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 15:18:48.554409 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 15:18:48.555528 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:18:48.555528 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:18:48.555528 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 15:18:48.555528 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:18:48.555528 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 15:18:48.555528 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:18:48.555528 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:18:48.555528 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:18:48.555528 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:18:48.555528 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:18:48.555528 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:18:48.555528 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:18:48.555528 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:18:48.555528 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:18:48.555528 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:18:48.555528 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:18:48.555528 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:18:48.555528 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:18:48.560584 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:18:48.560584 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:18:48.560584 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 15:18:48.560584 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 15:18:48.560584 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 15:18:48.560584 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 15:18:48.560584 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 15:18:48.560584 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 15:18:48.560584 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 07 15:18:48.560584 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:18:48.560584 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 607/1239] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:18:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:18:48.562090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a14da93-bb03-43b6-a9d6-f3b4114cb07f None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:48.660991 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a14da93-bb03-43b6-a9d6-f3b4114cb07f None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:48.694287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d696e-d810-4778-8d36-133ff5ccae05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:48.694952 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d696e-d810-4778-8d36-133ff5ccae05 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:48.695235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d696e-d810-4778-8d36-133ff5ccae05 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:48.696082 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d696e-d810-4778-8d36-133ff5ccae05 None admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:48.696929 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435d696e-d810-4778-8d36-133ff5ccae05 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:18:48.697215 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435d696e-d810-4778-8d36-133ff5ccae05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:48.698624 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435d696e-d810-4778-8d36-133ff5ccae05 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:48.715055 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 633/1240] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Jul 7 15:18:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:18:48.720884 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea64528-db5f-483b-b060-88b245c9b92f None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:48.822974 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea64528-db5f-483b-b060-88b245c9b92f None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:48.856327 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d68754e-97f7-42c8-8d4d-62ffca3c7aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:48.856991 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68754e-97f7-42c8-8d4d-62ffca3c7aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:48.857271 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68754e-97f7-42c8-8d4d-62ffca3c7aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:48.857543 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68754e-97f7-42c8-8d4d-62ffca3c7aa8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:48.916789 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:48.923085 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d68754e-97f7-42c8-8d4d-62ffca3c7aa8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:48.924553 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d68754e-97f7-42c8-8d4d-62ffca3c7aa8 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:49.022266 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 608/1241] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:18:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 07 15:18:49.028160 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d696e-d810-4778-8d36-133ff5ccae05 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:49.056326 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:49.135585 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d696e-d810-4778-8d36-133ff5ccae05 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:49.172772 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d826dbf5-e821-4506-8c36-82a703bd83d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:49.173565 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826dbf5-e821-4506-8c36-82a703bd83d5 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:49.173849 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826dbf5-e821-4506-8c36-82a703bd83d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:49.174158 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826dbf5-e821-4506-8c36-82a703bd83d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:49.185371 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d826dbf5-e821-4506-8c36-82a703bd83d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:49.190466 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d826dbf5-e821-4506-8c36-82a703bd83d5 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:49.204907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-d826dbf5-e821-4506-8c36-82a703bd83d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:18:49.204907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:18:49.204907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:18:49.204907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:18:49.204907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:18:49.204907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:18:49.204907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:18:49.204907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:18:49.204907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:18:49.204907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:18:49.204907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:18:49.204907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:18:49.204907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:18:49.204907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:18:49.204907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:18:49.205989 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:18:49.205989 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:18:49.205989 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:18:49.205989 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:18:49.205989 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:18:49.205989 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:18:49.205989 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:18:49.205989 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:18:49.205989 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:18:49.205989 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:18:49.205989 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:18:49.205989 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:18:49.205989 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:18:49.205989 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:18:49.205989 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:18:49.205989 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:18:49.205989 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:18:49.206966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:18:49.206966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:18:49.206966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:18:49.206966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:18:49.206966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 634/1242] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:18:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:18:49.212771 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d68754e-97f7-42c8-8d4d-62ffca3c7aa8 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:49.307094 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:49.312768 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d68754e-97f7-42c8-8d4d-62ffca3c7aa8 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:49.346740 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fcae45-ae89-474f-9ffc-a34a064fd1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:49.347391 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fcae45-ae89-474f-9ffc-a34a064fd1ab None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:49.347706 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fcae45-ae89-474f-9ffc-a34a064fd1ab None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:49.347985 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fcae45-ae89-474f-9ffc-a34a064fd1ab None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:49.348704 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fcae45-ae89-474f-9ffc-a34a064fd1ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:18:49.349012 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fcae45-ae89-474f-9ffc-a34a064fd1ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:49.350422 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fcae45-ae89-474f-9ffc-a34a064fd1ab None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:49.361389 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 609/1243] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:18:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:18:49.367086 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d826dbf5-e821-4506-8c36-82a703bd83d5 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:49.447060 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:49.468116 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d826dbf5-e821-4506-8c36-82a703bd83d5 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:49.501842 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c45c6-8665-4529-9d69-74a01b4fc39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:49.502685 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c45c6-8665-4529-9d69-74a01b4fc39f None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:49.502971 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c45c6-8665-4529-9d69-74a01b4fc39f None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:49.503246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c45c6-8665-4529-9d69-74a01b4fc39f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:49.509886 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356c45c6-8665-4529-9d69-74a01b4fc39f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:49.511320 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356c45c6-8665-4529-9d69-74a01b4fc39f None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:49.525293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-356c45c6-8665-4529-9d69-74a01b4fc39f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:18:49.525293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:18:49.525293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:18:49.525293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:18:49.525293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:18:49.525293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:18:49.525293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:18:49.525293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:18:49.525293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:18:49.525293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:18:49.525293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:18:49.525293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:18:49.525293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 15:18:49.525293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 15:18:49.525293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 15:18:49.526387 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 15:18:49.526387 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:18:49.526387 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:18:49.526387 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:18:49.526387 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:18:49.526387 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:18:49.526387 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:18:49.526387 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:18:49.526387 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:18:49.526387 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:18:49.526387 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:18:49.526387 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:18:49.526387 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:18:49.526387 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:18:49.526387 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:18:49.526387 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:18:49.526387 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:18:49.527335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 15:18:49.527335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 15:18:49.527335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 15:18:49.527335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 15:18:49.527335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 15:18:49.527335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 15:18:49.527335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:18:49.527335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:18:49.527335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 635/1244] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:18:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:18:49.532211 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fcae45-ae89-474f-9ffc-a34a064fd1ab None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:49.631973 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fcae45-ae89-474f-9ffc-a34a064fd1ab None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:49.665281 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227938e-2b71-4e01-854d-e4e76fc8baac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:49.665929 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227938e-2b71-4e01-854d-e4e76fc8baac None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:49.666211 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227938e-2b71-4e01-854d-e4e76fc8baac None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:49.666486 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227938e-2b71-4e01-854d-e4e76fc8baac None admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:49.667262 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d227938e-2b71-4e01-854d-e4e76fc8baac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:18:49.667598 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d227938e-2b71-4e01-854d-e4e76fc8baac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:49.669016 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d227938e-2b71-4e01-854d-e4e76fc8baac None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:49.680809 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 610/1245] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Jul 7 15:18:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:18:49.686934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c45c6-8665-4529-9d69-74a01b4fc39f None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:49.786030 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c45c6-8665-4529-9d69-74a01b4fc39f None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:49.818263 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9026a-17ba-4d38-8b4b-1b6d05d38bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:49.818905 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9026a-17ba-4d38-8b4b-1b6d05d38bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:49.819190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9026a-17ba-4d38-8b4b-1b6d05d38bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:49.819466 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9026a-17ba-4d38-8b4b-1b6d05d38bd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:49.820850 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb9026a-17ba-4d38-8b4b-1b6d05d38bd8 None admin] RBAC: Adding query filter params (role.id=3d4f4e0a472e4ff9ac68d90c8612e956, scope.project.id=c5260c7fd3614e9e93b167c86a94260e, user.id=755c2782cfb042f89950b8fc862e4fa1) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:18:49.821160 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb9026a-17ba-4d38-8b4b-1b6d05d38bd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:49.822577 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb9026a-17ba-4d38-8b4b-1b6d05d38bd8 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:49.828222 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 636/1246] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:18:49 2022] GET /identity/v3/role_assignments?user.id=755c2782cfb042f89950b8fc862e4fa1&scope.project.id=c5260c7fd3614e9e93b167c86a94260e&role.id=3d4f4e0a472e4ff9ac68d90c8612e956 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:18:51.888867 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a15f43-2e19-486c-ba46-d282397d72b9 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:51.889442 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a15f43-2e19-486c-ba46-d282397d72b9 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:51.889442 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a15f43-2e19-486c-ba46-d282397d72b9 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:51.890781 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 611/1247] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:18:51.895878 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66eb5ae-d763-49b0-84be-aba64fdac902 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:51.897942 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66eb5ae-d763-49b0-84be-aba64fdac902 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:51.897942 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66eb5ae-d763-49b0-84be-aba64fdac902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:51.902225 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:51.914564 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-d66eb5ae-d763-49b0-84be-aba64fdac902 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:18:51.915774 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-d66eb5ae-d763-49b0-84be-aba64fdac902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:18:51.981761 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 637/1248] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:18:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:18:52.004871 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a15f43-2e19-486c-ba46-d282397d72b9 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:52.023162 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:52.106089 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a15f43-2e19-486c-ba46-d282397d72b9 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:52.139371 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3118fc-6a17-44ae-8ae2-673cae022204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:52.140108 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3118fc-6a17-44ae-8ae2-673cae022204 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:52.140390 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3118fc-6a17-44ae-8ae2-673cae022204 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:52.140664 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3118fc-6a17-44ae-8ae2-673cae022204 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:52.151061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3118fc-6a17-44ae-8ae2-673cae022204 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:18:52.158642 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-dc3118fc-6a17-44ae-8ae2-673cae022204 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:18:52.158642 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:18:52.158642 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:18:52.158642 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:18:52.158642 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:18:52.158642 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:18:52.158642 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:18:52.158642 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:18:52.158642 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:18:52.158642 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:18:52.158642 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:18:52.158642 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:18:52.158642 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:18:52.158642 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:18:52.158642 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:18:52.158642 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:18:52.159859 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:18:52.159859 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:18:52.159859 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:18:52.159859 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:18:52.159859 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:18:52.159859 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:18:52.159859 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:18:52.159859 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:18:52.159859 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:18:52.159859 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:18:52.159859 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:18:52.159859 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:18:52.159859 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:18:52.159859 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:18:52.159859 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:18:52.159859 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:18:52.164472 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:18:52.164472 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:18:52.164472 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:18:52.164472 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:18:52.164472 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:18:52.164472 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:18:52.164472 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:18:52.164472 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:18:52.164472 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:18:52.164472 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:18:52.164472 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:18:52.164472 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:18:52.164472 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 612/1249] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Jul 7 15:18:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:18:52.166329 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66eb5ae-d763-49b0-84be-aba64fdac902 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:52.264839 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66eb5ae-d763-49b0-84be-aba64fdac902 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:52.293065 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:52.297817 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515cfa07-33ae-4068-bdba-8302ebb8415f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:52.298483 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515cfa07-33ae-4068-bdba-8302ebb8415f None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:52.298765 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515cfa07-33ae-4068-bdba-8302ebb8415f None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:52.299040 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515cfa07-33ae-4068-bdba-8302ebb8415f None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:52.299812 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515cfa07-33ae-4068-bdba-8302ebb8415f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:18:52.300119 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515cfa07-33ae-4068-bdba-8302ebb8415f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:52.301512 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515cfa07-33ae-4068-bdba-8302ebb8415f None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:52.309308 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 638/1250] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:18:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:18:52.315109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3118fc-6a17-44ae-8ae2-673cae022204 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:52.391458 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:52.414101 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3118fc-6a17-44ae-8ae2-673cae022204 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:52.447520 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4749ffae-46c6-41f9-aef5-8e4f2d1685c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:52.448326 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749ffae-46c6-41f9-aef5-8e4f2d1685c7 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:52.448610 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749ffae-46c6-41f9-aef5-8e4f2d1685c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:52.448887 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749ffae-46c6-41f9-aef5-8e4f2d1685c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:52.459226 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4749ffae-46c6-41f9-aef5-8e4f2d1685c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:52.460710 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4749ffae-46c6-41f9-aef5-8e4f2d1685c7 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:52.478061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-4749ffae-46c6-41f9-aef5-8e4f2d1685c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:18:52.478061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:18:52.478061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:18:52.478061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:18:52.478061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:18:52.478061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:18:52.478061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:18:52.478061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:18:52.478061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:18:52.478061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:18:52.478061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:18:52.478061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:18:52.478061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:18:52.478061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:18:52.478061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:18:52.479126 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:18:52.479126 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:18:52.479126 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:18:52.479126 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:18:52.479126 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:18:52.479126 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:18:52.479126 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:18:52.479126 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:18:52.479126 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:18:52.479126 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:18:52.479126 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:18:52.479126 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:18:52.479126 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:18:52.479126 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:18:52.479126 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:18:52.479126 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:18:52.479126 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:18:52.480344 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:18:52.480344 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:18:52.480344 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:18:52.480344 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:18:52.480344 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 613/1251] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:18:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:18:52.484925 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515cfa07-33ae-4068-bdba-8302ebb8415f None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:52.583668 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515cfa07-33ae-4068-bdba-8302ebb8415f None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:52.619092 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72364e91-1b0d-41b0-9762-1c1d256b432d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:52.619774 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72364e91-1b0d-41b0-9762-1c1d256b432d None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:52.620058 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72364e91-1b0d-41b0-9762-1c1d256b432d None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:52.620335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72364e91-1b0d-41b0-9762-1c1d256b432d None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:52.621046 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72364e91-1b0d-41b0-9762-1c1d256b432d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:18:52.621353 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72364e91-1b0d-41b0-9762-1c1d256b432d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:52.622712 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72364e91-1b0d-41b0-9762-1c1d256b432d None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:52.634577 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 639/1252] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:18:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:18:52.640186 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4749ffae-46c6-41f9-aef5-8e4f2d1685c7 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:52.739099 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4749ffae-46c6-41f9-aef5-8e4f2d1685c7 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:52.765061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:52.772550 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67ffa98-3f89-4678-995e-47e3995f7d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:52.773357 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ffa98-3f89-4678-995e-47e3995f7d3b None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:52.773637 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ffa98-3f89-4678-995e-47e3995f7d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:52.773908 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ffa98-3f89-4678-995e-47e3995f7d3b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:52.781152 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67ffa98-3f89-4678-995e-47e3995f7d3b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:52.782558 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67ffa98-3f89-4678-995e-47e3995f7d3b None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:52.797671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-a67ffa98-3f89-4678-995e-47e3995f7d3b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 07 15:18:52.797671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:18:52.797671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:18:52.797671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:18:52.797671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:18:52.797671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:18:52.797671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:18:52.797671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:18:52.797671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:18:52.797671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:18:52.797671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:18:52.797671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:18:52.797671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 15:18:52.797671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 15:18:52.797671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 15:18:52.797671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 15:18:52.798857 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:18:52.798857 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:18:52.798857 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 15:18:52.798857 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:18:52.798857 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 15:18:52.798857 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:18:52.798857 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:18:52.798857 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:18:52.798857 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:18:52.798857 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:18:52.798857 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:18:52.798857 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:18:52.798857 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:18:52.798857 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:18:52.798857 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:18:52.798857 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:18:52.798857 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:18:52.798857 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:18:52.800226 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:18:52.800226 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:18:52.800226 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 15:18:52.800226 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 15:18:52.800226 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 15:18:52.800226 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 15:18:52.800226 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 15:18:52.800226 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 15:18:52.800226 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 07 15:18:52.800226 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:18:52.800226 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 614/1253] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:18:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:18:52.805416 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72364e91-1b0d-41b0-9762-1c1d256b432d None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:52.860386 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:52.902878 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72364e91-1b0d-41b0-9762-1c1d256b432d None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:52.937692 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bfe0c5-d7bd-4d90-bd49-163a793ec00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:52.938361 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bfe0c5-d7bd-4d90-bd49-163a793ec00e None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:52.938644 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bfe0c5-d7bd-4d90-bd49-163a793ec00e None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:52.938921 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bfe0c5-d7bd-4d90-bd49-163a793ec00e None admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:52.939836 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bfe0c5-d7bd-4d90-bd49-163a793ec00e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:18:52.943580 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bfe0c5-d7bd-4d90-bd49-163a793ec00e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:52.943580 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bfe0c5-d7bd-4d90-bd49-163a793ec00e None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:52.958986 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 640/1254] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Jul 7 15:18:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:18:52.964692 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67ffa98-3f89-4678-995e-47e3995f7d3b None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:53.063520 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67ffa98-3f89-4678-995e-47e3995f7d3b None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:53.095259 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd11d8-f037-4ed9-a431-1c497c731ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:53.095920 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd11d8-f037-4ed9-a431-1c497c731ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:53.096466 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd11d8-f037-4ed9-a431-1c497c731ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:53.096741 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd11d8-f037-4ed9-a431-1c497c731ae8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:53.152074 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:53.164753 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dd11d8-f037-4ed9-a431-1c497c731ae8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:53.166165 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dd11d8-f037-4ed9-a431-1c497c731ae8 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:53.262016 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 615/1255] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:18:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 07 15:18:53.267873 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bfe0c5-d7bd-4d90-bd49-163a793ec00e None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:53.364403 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bfe0c5-d7bd-4d90-bd49-163a793ec00e None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:53.377330 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:53.397323 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8882f-e11b-42ec-bbfd-517197d22b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:53.398101 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8882f-e11b-42ec-bbfd-517197d22b35 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:53.398893 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8882f-e11b-42ec-bbfd-517197d22b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:53.399164 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8882f-e11b-42ec-bbfd-517197d22b35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:53.408966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d8882f-e11b-42ec-bbfd-517197d22b35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:53.410398 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d8882f-e11b-42ec-bbfd-517197d22b35 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:53.427775 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-55d8882f-e11b-42ec-bbfd-517197d22b35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:18:53.427775 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:18:53.427775 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:18:53.427775 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:18:53.427775 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:18:53.427775 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:18:53.427775 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:18:53.427775 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:18:53.427775 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:18:53.427775 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:18:53.427775 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:18:53.427775 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:18:53.427775 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:18:53.427775 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:18:53.427775 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:18:53.429816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:18:53.429816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:18:53.429816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:18:53.429816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:18:53.429816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:18:53.429816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:18:53.429816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:18:53.429816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:18:53.429816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:18:53.429816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:18:53.429816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:18:53.429816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:18:53.429816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:18:53.429816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:18:53.429816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:18:53.429816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:18:53.429816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:18:53.430939 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:18:53.430939 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:18:53.430939 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:18:53.430939 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:18:53.430939 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 641/1256] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:18:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:18:53.434765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd11d8-f037-4ed9-a431-1c497c731ae8 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:53.520564 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:53.532673 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd11d8-f037-4ed9-a431-1c497c731ae8 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:53.565622 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcca7d9-92ac-418c-9944-8c270d37ee04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:53.566272 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcca7d9-92ac-418c-9944-8c270d37ee04 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:53.566555 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcca7d9-92ac-418c-9944-8c270d37ee04 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:53.566829 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcca7d9-92ac-418c-9944-8c270d37ee04 None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:53.567544 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcca7d9-92ac-418c-9944-8c270d37ee04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:18:53.567877 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcca7d9-92ac-418c-9944-8c270d37ee04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:53.569217 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcca7d9-92ac-418c-9944-8c270d37ee04 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:53.580906 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 616/1257] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:18:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:18:53.587122 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8882f-e11b-42ec-bbfd-517197d22b35 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:53.699000 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8882f-e11b-42ec-bbfd-517197d22b35 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:53.748048 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a4a9e0-17cf-467a-b425-7b0318f54e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:53.749214 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4a9e0-17cf-467a-b425-7b0318f54e68 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:53.749628 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4a9e0-17cf-467a-b425-7b0318f54e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:53.750007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4a9e0-17cf-467a-b425-7b0318f54e68 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:53.761826 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a4a9e0-17cf-467a-b425-7b0318f54e68 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:53.763295 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a4a9e0-17cf-467a-b425-7b0318f54e68 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:53.778442 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-87a4a9e0-17cf-467a-b425-7b0318f54e68 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:18:53.778442 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:18:53.778442 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:18:53.778442 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:18:53.778442 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:18:53.778442 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:18:53.778442 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:18:53.778442 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:18:53.778442 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:18:53.778442 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:18:53.778442 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:18:53.778442 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:18:53.778442 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 15:18:53.778442 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 15:18:53.778442 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 15:18:53.779556 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 15:18:53.779556 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:18:53.779556 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:18:53.779556 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:18:53.779556 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:18:53.779556 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:18:53.779556 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:18:53.779556 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:18:53.779556 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:18:53.779556 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:18:53.779556 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:18:53.779556 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:18:53.779556 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:18:53.779556 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:18:53.779556 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:18:53.779556 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:18:53.779556 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:18:53.795866 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 15:18:53.795866 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 15:18:53.795866 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 15:18:53.795866 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 15:18:53.795866 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 15:18:53.795866 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 15:18:53.795866 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:18:53.795866 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:18:53.795866 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 642/1258] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:18:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:18:53.796476 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcca7d9-92ac-418c-9944-8c270d37ee04 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:53.893087 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcca7d9-92ac-418c-9944-8c270d37ee04 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:53.928079 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b62959e-b80f-4933-8fd4-143752d148d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:53.928982 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62959e-b80f-4933-8fd4-143752d148d5 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:53.929267 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62959e-b80f-4933-8fd4-143752d148d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:53.930052 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62959e-b80f-4933-8fd4-143752d148d5 None admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:53.930834 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b62959e-b80f-4933-8fd4-143752d148d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:18:53.931163 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b62959e-b80f-4933-8fd4-143752d148d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:53.932635 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b62959e-b80f-4933-8fd4-143752d148d5 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:53.946037 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 617/1259] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Jul 7 15:18:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:18:53.952511 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a4a9e0-17cf-467a-b425-7b0318f54e68 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:53.979272 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:54.055720 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a4a9e0-17cf-467a-b425-7b0318f54e68 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:54.091088 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c18a7-92c5-48ae-8ecc-94cedddafc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:54.091775 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c18a7-92c5-48ae-8ecc-94cedddafc87 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:54.092058 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c18a7-92c5-48ae-8ecc-94cedddafc87 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:54.092333 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c18a7-92c5-48ae-8ecc-94cedddafc87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:54.093165 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c18a7-92c5-48ae-8ecc-94cedddafc87 None admin] RBAC: Adding query filter params (role.id=0e7793fbf3064037984da651879e3a94, scope.project.id=c5260c7fd3614e9e93b167c86a94260e, user.id=755c2782cfb042f89950b8fc862e4fa1) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:18:54.093473 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c18a7-92c5-48ae-8ecc-94cedddafc87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:54.094899 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c18a7-92c5-48ae-8ecc-94cedddafc87 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:54.100986 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 643/1260] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:18:53 2022] GET /identity/v3/role_assignments?user.id=755c2782cfb042f89950b8fc862e4fa1&scope.project.id=c5260c7fd3614e9e93b167c86a94260e&role.id=0e7793fbf3064037984da651879e3a94 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:18:56.087207 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f632550d-671e-483f-a85a-d83f5b88977b None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:56.087778 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f632550d-671e-483f-a85a-d83f5b88977b None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:56.087910 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f632550d-671e-483f-a85a-d83f5b88977b None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:56.089167 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 618/1261] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:18:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:18:56.094572 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f73a97-ba1d-45b7-bb07-38cc89e67079 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:56.094975 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f73a97-ba1d-45b7-bb07-38cc89e67079 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:56.095158 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f73a97-ba1d-45b7-bb07-38cc89e67079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:56.112253 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-04f73a97-ba1d-45b7-bb07-38cc89e67079 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:18:56.113437 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-04f73a97-ba1d-45b7-bb07-38cc89e67079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:18:56.184282 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 644/1262] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:18:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:18:56.207110 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f632550d-671e-483f-a85a-d83f5b88977b None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:56.218045 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:56.306352 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f632550d-671e-483f-a85a-d83f5b88977b None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:56.339512 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aedc13d-3a58-4cc7-8dc3-f8c079dfd44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:56.340328 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedc13d-3a58-4cc7-8dc3-f8c079dfd44b None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:56.340610 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedc13d-3a58-4cc7-8dc3-f8c079dfd44b None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:56.340881 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedc13d-3a58-4cc7-8dc3-f8c079dfd44b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:56.351786 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aedc13d-3a58-4cc7-8dc3-f8c079dfd44b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:18:56.359300 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-3aedc13d-3a58-4cc7-8dc3-f8c079dfd44b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:18:56.359300 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:18:56.359300 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:18:56.359300 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:18:56.359300 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:18:56.359300 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:18:56.359300 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:18:56.359300 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:18:56.359300 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:18:56.359300 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:18:56.359300 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:18:56.359300 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:18:56.359300 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:18:56.359300 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:18:56.359300 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:18:56.359300 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:18:56.360438 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:18:56.360438 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:18:56.360438 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:18:56.360438 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:18:56.360438 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:18:56.360438 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:18:56.360438 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:18:56.360438 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:18:56.360438 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:18:56.360438 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:18:56.360438 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:18:56.360438 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:18:56.360438 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:18:56.360438 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:18:56.360438 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:18:56.360438 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:18:56.361474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:18:56.361474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:18:56.361474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:18:56.361474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:18:56.361474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:18:56.361474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:18:56.361474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:18:56.361474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:18:56.361474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:18:56.361474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:18:56.361474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:18:56.361474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:18:56.361474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 619/1263] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Jul 7 15:18:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:18:56.366800 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f73a97-ba1d-45b7-bb07-38cc89e67079 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:56.384555 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:56.465424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f73a97-ba1d-45b7-bb07-38cc89e67079 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:56.498866 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327e299-1cb8-4054-9dc8-4707df83039d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:56.499453 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327e299-1cb8-4054-9dc8-4707df83039d None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:56.499761 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327e299-1cb8-4054-9dc8-4707df83039d None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:56.500033 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327e299-1cb8-4054-9dc8-4707df83039d None admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:56.500768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3327e299-1cb8-4054-9dc8-4707df83039d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:18:56.501068 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3327e299-1cb8-4054-9dc8-4707df83039d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:56.502437 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3327e299-1cb8-4054-9dc8-4707df83039d None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:56.511016 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 645/1264] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:18:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:18:56.516771 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aedc13d-3a58-4cc7-8dc3-f8c079dfd44b None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:56.582949 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:56.615780 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aedc13d-3a58-4cc7-8dc3-f8c079dfd44b None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:56.648904 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a369512-01ff-479d-a0db-7fcb031c78f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:56.649672 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a369512-01ff-479d-a0db-7fcb031c78f2 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:56.649950 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a369512-01ff-479d-a0db-7fcb031c78f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:56.650220 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a369512-01ff-479d-a0db-7fcb031c78f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:56.660303 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a369512-01ff-479d-a0db-7fcb031c78f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:56.661728 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a369512-01ff-479d-a0db-7fcb031c78f2 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:56.678151 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-5a369512-01ff-479d-a0db-7fcb031c78f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:18:56.678151 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:18:56.678151 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:18:56.678151 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:18:56.678151 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:18:56.678151 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:18:56.678151 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:18:56.678151 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:18:56.678151 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:18:56.678151 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:18:56.678151 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:18:56.678151 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:18:56.678151 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:18:56.678151 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:18:56.678151 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:18:56.679223 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:18:56.679223 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:18:56.679223 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:18:56.679223 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:18:56.679223 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:18:56.679223 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:18:56.679223 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:18:56.679223 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:18:56.679223 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:18:56.679223 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:18:56.679223 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:18:56.679223 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:18:56.679223 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:18:56.679223 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:18:56.679223 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:18:56.679223 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:18:56.679223 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:18:56.680460 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:18:56.680460 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:18:56.680460 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:18:56.680460 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:18:56.683730 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 620/1265] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:18:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:18:56.685117 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327e299-1cb8-4054-9dc8-4707df83039d None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:56.765932 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:56.785060 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327e299-1cb8-4054-9dc8-4707df83039d None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:56.824173 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce291a05-f0f1-4b95-af4c-00b689397320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:56.824836 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce291a05-f0f1-4b95-af4c-00b689397320 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:56.825111 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce291a05-f0f1-4b95-af4c-00b689397320 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:56.825379 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce291a05-f0f1-4b95-af4c-00b689397320 None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:56.826103 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce291a05-f0f1-4b95-af4c-00b689397320 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:18:56.826403 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce291a05-f0f1-4b95-af4c-00b689397320 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:56.827948 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce291a05-f0f1-4b95-af4c-00b689397320 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:56.840734 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 646/1266] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:18:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:18:56.846930 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a369512-01ff-479d-a0db-7fcb031c78f2 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:56.947480 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a369512-01ff-479d-a0db-7fcb031c78f2 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:56.967120 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:56.981064 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a024b-12d8-4c61-a39b-dda0f5115dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:56.981850 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a024b-12d8-4c61-a39b-dda0f5115dad None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:56.982176 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a024b-12d8-4c61-a39b-dda0f5115dad None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:56.982449 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a024b-12d8-4c61-a39b-dda0f5115dad None admin] PATH_INFO: `/v3/users/placement` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:56.994599 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6a024b-12d8-4c61-a39b-dda0f5115dad None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:56.996053 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6a024b-12d8-4c61-a39b-dda0f5115dad None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:57.011157 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-9c6a024b-12d8-4c61-a39b-dda0f5115dad None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 07 15:18:57.011157 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:18:57.011157 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:18:57.011157 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:18:57.011157 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:18:57.011157 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:18:57.011157 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:18:57.011157 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:18:57.011157 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:18:57.011157 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:18:57.011157 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:18:57.011157 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:18:57.011157 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 15:18:57.011157 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 15:18:57.011157 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 15:18:57.011157 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 15:18:57.012329 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:18:57.012329 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:18:57.012329 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 15:18:57.012329 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:18:57.012329 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 15:18:57.012329 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:18:57.012329 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:18:57.012329 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:18:57.012329 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:18:57.012329 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:18:57.012329 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:18:57.012329 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:18:57.012329 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:18:57.012329 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:18:57.012329 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:18:57.012329 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:18:57.012329 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:18:57.012329 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:18:57.013541 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:18:57.013541 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:18:57.013541 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 15:18:57.013541 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 15:18:57.013541 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 15:18:57.013541 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 15:18:57.013541 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 15:18:57.013541 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 15:18:57.013541 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 07 15:18:57.013541 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:18:57.013541 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 621/1267] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:18:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:18:57.018337 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce291a05-f0f1-4b95-af4c-00b689397320 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:57.118521 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce291a05-f0f1-4b95-af4c-00b689397320 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:57.152233 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7052d939-3262-45a4-b97c-9053712f3254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:57.152894 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052d939-3262-45a4-b97c-9053712f3254 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:57.153175 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052d939-3262-45a4-b97c-9053712f3254 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:57.153448 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052d939-3262-45a4-b97c-9053712f3254 None admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:57.154281 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7052d939-3262-45a4-b97c-9053712f3254 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:18:57.154586 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7052d939-3262-45a4-b97c-9053712f3254 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:57.156043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7052d939-3262-45a4-b97c-9053712f3254 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:57.164398 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:57.172462 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 647/1268] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Jul 7 15:18:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:18:57.178271 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a024b-12d8-4c61-a39b-dda0f5115dad None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:57.279551 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a024b-12d8-4c61-a39b-dda0f5115dad None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:57.312603 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56d1b0-fb12-4d6a-b002-6f7547392412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:57.313460 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56d1b0-fb12-4d6a-b002-6f7547392412 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:57.313746 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56d1b0-fb12-4d6a-b002-6f7547392412 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:57.314021 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56d1b0-fb12-4d6a-b002-6f7547392412 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:57.324269 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb56d1b0-fb12-4d6a-b002-6f7547392412 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:57.325703 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb56d1b0-fb12-4d6a-b002-6f7547392412 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:57.342268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-eb56d1b0-fb12-4d6a-b002-6f7547392412 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:18:57.342268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:18:57.342268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:18:57.342268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:18:57.342268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:18:57.342268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:18:57.342268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:18:57.342268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:18:57.342268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:18:57.342268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:18:57.342268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:18:57.342268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:18:57.342268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:18:57.342268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:18:57.342268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:18:57.347943 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:18:57.347943 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:18:57.347943 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:18:57.347943 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:18:57.347943 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:18:57.347943 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:18:57.347943 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:18:57.347943 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:18:57.347943 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:18:57.347943 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:18:57.347943 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:18:57.347943 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:18:57.347943 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:18:57.347943 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:18:57.347943 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:18:57.347943 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:18:57.347943 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:18:57.349099 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:18:57.349099 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:18:57.349099 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:18:57.349099 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:18:57.349099 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 622/1269] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:18:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:18:57.349542 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7052d939-3262-45a4-b97c-9053712f3254 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:57.448832 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7052d939-3262-45a4-b97c-9053712f3254 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:57.482263 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af0257-7880-45e6-af64-8d75bdf13874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:57.482918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af0257-7880-45e6-af64-8d75bdf13874 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:57.483196 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af0257-7880-45e6-af64-8d75bdf13874 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:57.483469 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af0257-7880-45e6-af64-8d75bdf13874 None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:57.484215 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5af0257-7880-45e6-af64-8d75bdf13874 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:18:57.484523 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5af0257-7880-45e6-af64-8d75bdf13874 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:57.485881 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5af0257-7880-45e6-af64-8d75bdf13874 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:57.497326 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 648/1270] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:18:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:18:57.503141 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56d1b0-fb12-4d6a-b002-6f7547392412 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:57.535372 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:57.603271 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56d1b0-fb12-4d6a-b002-6f7547392412 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:57.637400 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30fd3b-ddbc-4e7f-b24c-feae996ec707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:57.638172 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30fd3b-ddbc-4e7f-b24c-feae996ec707 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:57.638454 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30fd3b-ddbc-4e7f-b24c-feae996ec707 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:57.638726 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30fd3b-ddbc-4e7f-b24c-feae996ec707 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:57.645325 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa30fd3b-ddbc-4e7f-b24c-feae996ec707 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:57.646822 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa30fd3b-ddbc-4e7f-b24c-feae996ec707 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:57.662360 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-fa30fd3b-ddbc-4e7f-b24c-feae996ec707 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:18:57.662360 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:18:57.662360 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:18:57.662360 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:18:57.662360 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:18:57.662360 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:18:57.662360 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:18:57.662360 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:18:57.662360 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:18:57.662360 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:18:57.662360 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:18:57.662360 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:18:57.662360 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 15:18:57.662360 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 15:18:57.662360 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 15:18:57.664199 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 15:18:57.664199 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:18:57.664199 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:18:57.664199 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:18:57.664199 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:18:57.664199 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:18:57.664199 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:18:57.664199 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:18:57.664199 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:18:57.664199 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:18:57.664199 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:18:57.664199 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:18:57.664199 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:18:57.664199 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:18:57.664199 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:18:57.664199 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:18:57.664199 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:18:57.665303 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 15:18:57.665303 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 15:18:57.665303 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 15:18:57.665303 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 15:18:57.665303 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 15:18:57.665303 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 15:18:57.665303 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:18:57.665303 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:18:57.665303 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 623/1271] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:18:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:18:57.669951 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af0257-7880-45e6-af64-8d75bdf13874 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:57.738462 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:57.778804 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af0257-7880-45e6-af64-8d75bdf13874 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:57.813114 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e9e99-7b50-488c-bfa8-95c1e42a5c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:57.813710 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e9e99-7b50-488c-bfa8-95c1e42a5c6a None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:57.813995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e9e99-7b50-488c-bfa8-95c1e42a5c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:57.814270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e9e99-7b50-488c-bfa8-95c1e42a5c6a None admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:57.815059 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e9e99-7b50-488c-bfa8-95c1e42a5c6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:18:57.815369 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e9e99-7b50-488c-bfa8-95c1e42a5c6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:57.816837 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e9e99-7b50-488c-bfa8-95c1e42a5c6a None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:57.829136 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 649/1272] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Jul 7 15:18:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:18:57.835095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30fd3b-ddbc-4e7f-b24c-feae996ec707 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:18:57.924441 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:18:57.932751 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30fd3b-ddbc-4e7f-b24c-feae996ec707 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:18:57.965668 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b78f4-8bd3-4b17-967e-3548072167b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:18:57.966583 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b78f4-8bd3-4b17-967e-3548072167b3 None admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:18:57.966861 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b78f4-8bd3-4b17-967e-3548072167b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:18:57.967132 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b78f4-8bd3-4b17-967e-3548072167b3 None admin] PATH_INFO: `/v3/projects/c5260c7fd3614e9e93b167c86a94260e/users/755c2782cfb042f89950b8fc862e4fa1/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:18:57.969062 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74b78f4-8bd3-4b17-967e-3548072167b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=c5260c7fd3614e9e93b167c86a94260e, user_id=755c2782cfb042f89950b8fc862e4fa1, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:18:57.970478 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74b78f4-8bd3-4b17-967e-3548072167b3 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:18:58.088528 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 624/1273] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Jul 7 15:18:57 2022] PUT /identity/v3/projects/c5260c7fd3614e9e93b167c86a94260e/users/755c2782cfb042f89950b8fc862e4fa1/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:19:00.078910 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceeca8a-2d52-4b5b-bda9-0fdbbaf4703c None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:19:00.079431 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceeca8a-2d52-4b5b-bda9-0fdbbaf4703c None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:19:00.079581 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceeca8a-2d52-4b5b-bda9-0fdbbaf4703c None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:19:00.080944 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 650/1274] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:19:00.086253 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3543a81-3d4d-4419-9fdb-60aa467458f8 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:19:00.086664 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3543a81-3d4d-4419-9fdb-60aa467458f8 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:19:00.086891 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3543a81-3d4d-4419-9fdb-60aa467458f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:19:00.103606 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-f3543a81-3d4d-4419-9fdb-60aa467458f8 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:19:00.104813 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-f3543a81-3d4d-4419-9fdb-60aa467458f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:19:00.171143 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 625/1275] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:19:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:19:00.194582 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceeca8a-2d52-4b5b-bda9-0fdbbaf4703c None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:19:00.294452 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceeca8a-2d52-4b5b-bda9-0fdbbaf4703c None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:19:00.310802 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:19:00.327155 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98844b2e-2176-448c-bbbe-07137efbd680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:19:00.327966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98844b2e-2176-448c-bbbe-07137efbd680 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:19:00.328251 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98844b2e-2176-448c-bbbe-07137efbd680 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:19:00.328522 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98844b2e-2176-448c-bbbe-07137efbd680 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:19:00.338847 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98844b2e-2176-448c-bbbe-07137efbd680 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 15:19:00.346696 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-98844b2e-2176-448c-bbbe-07137efbd680 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:19:00.346696 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:19:00.346696 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:19:00.346696 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:19:00.346696 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:19:00.346696 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:19:00.346696 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:19:00.346696 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:19:00.346696 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:19:00.346696 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:19:00.346696 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:19:00.346696 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:19:00.346696 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 15:19:00.346696 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 15:19:00.346696 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 15:19:00.346696 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 15:19:00.347842 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 15:19:00.347842 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 15:19:00.347842 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 15:19:00.347842 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 15:19:00.347842 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:19:00.347842 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:19:00.347842 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:19:00.347842 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:19:00.347842 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:19:00.347842 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:19:00.347842 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:19:00.347842 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:19:00.347842 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:19:00.347842 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:19:00.347842 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:19:00.347842 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:19:00.348899 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:19:00.348899 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:19:00.348899 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:19:00.348899 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:19:00.348899 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 15:19:00.348899 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 15:19:00.348899 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 15:19:00.348899 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 15:19:00.348899 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 15:19:00.348899 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 15:19:00.348899 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 15:19:00.348899 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:19:00.348899 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 651/1276] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Jul 7 15:19:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:19:00.353833 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3543a81-3d4d-4419-9fdb-60aa467458f8 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:19:00.433144 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:19:00.450748 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3543a81-3d4d-4419-9fdb-60aa467458f8 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:19:00.483425 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac747558-4c15-47a1-b508-204aca3d85e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:19:00.484127 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac747558-4c15-47a1-b508-204aca3d85e8 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:19:00.484405 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac747558-4c15-47a1-b508-204aca3d85e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:19:00.484675 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac747558-4c15-47a1-b508-204aca3d85e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:19:00.485417 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac747558-4c15-47a1-b508-204aca3d85e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:19:00.485717 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac747558-4c15-47a1-b508-204aca3d85e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:19:00.487055 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac747558-4c15-47a1-b508-204aca3d85e8 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:19:00.495717 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 626/1277] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Jul 7 15:19:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:19:00.501234 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98844b2e-2176-448c-bbbe-07137efbd680 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:19:00.598841 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98844b2e-2176-448c-bbbe-07137efbd680 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:19:00.631242 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3740e3b-cae3-48a3-be6b-6ac562a1d155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:19:00.632563 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3740e3b-cae3-48a3-be6b-6ac562a1d155 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:19:00.632839 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3740e3b-cae3-48a3-be6b-6ac562a1d155 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:19:00.633110 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3740e3b-cae3-48a3-be6b-6ac562a1d155 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:19:00.643312 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3740e3b-cae3-48a3-be6b-6ac562a1d155 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:19:00.644783 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3740e3b-cae3-48a3-be6b-6ac562a1d155 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:19:00.662254 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-f3740e3b-cae3-48a3-be6b-6ac562a1d155 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:19:00.662254 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:19:00.662254 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:19:00.662254 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:19:00.662254 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:19:00.662254 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:19:00.662254 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:19:00.662254 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:19:00.662254 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:19:00.662254 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:19:00.662254 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:19:00.662254 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:19:00.662254 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:19:00.662254 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:19:00.662254 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:19:00.663414 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:19:00.663414 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:19:00.663414 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:19:00.663414 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:19:00.663414 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:19:00.663414 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:19:00.663414 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:19:00.663414 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:19:00.663414 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:19:00.663414 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:19:00.663414 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:19:00.663414 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:19:00.663414 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:19:00.663414 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:19:00.663414 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:19:00.663414 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:19:00.663414 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:19:00.665766 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:19:00.665766 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:19:00.665766 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:19:00.665766 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:19:00.665766 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 652/1278] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:19:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:19:00.670174 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac747558-4c15-47a1-b508-204aca3d85e8 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:19:00.767468 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac747558-4c15-47a1-b508-204aca3d85e8 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:19:00.800027 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ead0d-7c57-4ef2-973a-ce7713ca1ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:19:00.800671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ead0d-7c57-4ef2-973a-ce7713ca1ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:19:00.800957 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ead0d-7c57-4ef2-973a-ce7713ca1ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:19:00.801231 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ead0d-7c57-4ef2-973a-ce7713ca1ea0 None admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:19:00.801960 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ead0d-7c57-4ef2-973a-ce7713ca1ea0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:19:00.802248 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ead0d-7c57-4ef2-973a-ce7713ca1ea0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:19:00.803627 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ead0d-7c57-4ef2-973a-ce7713ca1ea0 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:19:00.816286 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 627/1279] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:19:00.822492 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3740e3b-cae3-48a3-be6b-6ac562a1d155 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:19:00.858797 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:19:00.925294 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3740e3b-cae3-48a3-be6b-6ac562a1d155 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:19:00.958885 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d807405-7264-44dc-b97f-5910cab2950c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:19:00.959711 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d807405-7264-44dc-b97f-5910cab2950c None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:19:00.960004 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d807405-7264-44dc-b97f-5910cab2950c None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:19:00.960283 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d807405-7264-44dc-b97f-5910cab2950c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:19:00.967897 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d807405-7264-44dc-b97f-5910cab2950c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:19:00.969330 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d807405-7264-44dc-b97f-5910cab2950c None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:19:00.983750 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-3d807405-7264-44dc-b97f-5910cab2950c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 07 15:19:00.983750 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:19:00.983750 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:19:00.983750 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:19:00.983750 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:19:00.983750 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:19:00.983750 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:19:00.983750 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:19:00.983750 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:19:00.983750 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:19:00.983750 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:19:00.983750 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:19:00.983750 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 15:19:00.983750 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 15:19:00.983750 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 15:19:00.983750 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 15:19:00.984753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:19:00.984753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:19:00.984753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 15:19:00.984753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:19:00.984753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 15:19:00.984753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 15:19:00.984753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:19:00.984753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:19:00.984753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:19:00.984753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:19:00.984753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:19:00.984753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:19:00.984753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:19:00.984753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:19:00.984753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:19:00.984753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:19:00.984753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:19:00.984753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:19:00.985836 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:19:00.985836 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:19:00.985836 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 15:19:00.985836 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 15:19:00.985836 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 15:19:00.985836 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 15:19:00.985836 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 15:19:00.985836 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 15:19:00.985836 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 07 15:19:00.985836 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:19:00.985836 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 653/1280] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:19:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:19:00.990838 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ead0d-7c57-4ef2-973a-ce7713ca1ea0 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:19:00.995112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:19:01.088203 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ead0d-7c57-4ef2-973a-ce7713ca1ea0 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:19:01.122985 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f94c9-fd73-4572-9bfd-06ba31d35a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:19:01.123600 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f94c9-fd73-4572-9bfd-06ba31d35a90 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:19:01.123889 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f94c9-fd73-4572-9bfd-06ba31d35a90 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:19:01.124163 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f94c9-fd73-4572-9bfd-06ba31d35a90 None admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:19:01.124987 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2f94c9-fd73-4572-9bfd-06ba31d35a90 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:19:01.125292 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2f94c9-fd73-4572-9bfd-06ba31d35a90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:19:01.126797 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2f94c9-fd73-4572-9bfd-06ba31d35a90 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:19:01.143650 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 628/1281] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Jul 7 15:19:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:19:01.149863 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d807405-7264-44dc-b97f-5910cab2950c None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:19:01.239707 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:19:01.249507 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d807405-7264-44dc-b97f-5910cab2950c None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:19:01.281015 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e0eb8c-4175-4248-90fe-12d7ffe89f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:19:01.281645 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e0eb8c-4175-4248-90fe-12d7ffe89f67 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:19:01.281928 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e0eb8c-4175-4248-90fe-12d7ffe89f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:19:01.282205 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e0eb8c-4175-4248-90fe-12d7ffe89f67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:19:01.347665 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e0eb8c-4175-4248-90fe-12d7ffe89f67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:19:01.349110 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e0eb8c-4175-4248-90fe-12d7ffe89f67 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:19:01.435921 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:19:01.445704 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 654/1282] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Jul 7 15:19:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 07 15:19:01.451649 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f94c9-fd73-4572-9bfd-06ba31d35a90 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:19:01.513525 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:19:01.550452 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f94c9-fd73-4572-9bfd-06ba31d35a90 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:19:01.582916 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee86a7-3a91-46b1-9121-a29e55936fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:19:01.583772 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee86a7-3a91-46b1-9121-a29e55936fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:19:01.584342 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee86a7-3a91-46b1-9121-a29e55936fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:19:01.584698 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee86a7-3a91-46b1-9121-a29e55936fa8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:19:01.595789 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee86a7-3a91-46b1-9121-a29e55936fa8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:19:01.597479 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee86a7-3a91-46b1-9121-a29e55936fa8 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:19:01.614252 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-5bee86a7-3a91-46b1-9121-a29e55936fa8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:19:01.614252 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:19:01.614252 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:19:01.614252 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:19:01.614252 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:19:01.614252 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:19:01.614252 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:19:01.614252 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:19:01.614252 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:19:01.614252 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:19:01.614252 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:19:01.614252 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:19:01.614252 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 15:19:01.614252 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 15:19:01.614252 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 15:19:01.615335 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 15:19:01.615335 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:19:01.615335 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:19:01.615335 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:19:01.615335 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:19:01.615335 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:19:01.615335 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:19:01.615335 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:19:01.615335 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:19:01.615335 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:19:01.615335 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:19:01.615335 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:19:01.615335 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:19:01.615335 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:19:01.615335 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:19:01.615335 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:19:01.615335 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:19:01.616990 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 15:19:01.616990 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 15:19:01.616990 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 15:19:01.616990 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:19:01.616990 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 629/1283] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:19:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:19:01.621880 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e0eb8c-4175-4248-90fe-12d7ffe89f67 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:19:01.719643 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e0eb8c-4175-4248-90fe-12d7ffe89f67 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:19:01.753016 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd739a-64c1-47c8-a30b-aaeaae0c136b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:19:01.753666 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd739a-64c1-47c8-a30b-aaeaae0c136b None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:19:01.753948 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd739a-64c1-47c8-a30b-aaeaae0c136b None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:19:01.754221 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd739a-64c1-47c8-a30b-aaeaae0c136b None admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:19:01.754964 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd739a-64c1-47c8-a30b-aaeaae0c136b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:19:01.755345 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd739a-64c1-47c8-a30b-aaeaae0c136b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:19:01.756740 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd739a-64c1-47c8-a30b-aaeaae0c136b None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:19:01.768205 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 655/1284] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Jul 7 15:19:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:19:01.773874 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee86a7-3a91-46b1-9121-a29e55936fa8 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:19:01.873217 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee86a7-3a91-46b1-9121-a29e55936fa8 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:19:01.886759 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:19:01.905869 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82006b0-e040-4f12-b6fb-b6ac5c2fc6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:19:01.906637 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82006b0-e040-4f12-b6fb-b6ac5c2fc6a4 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:19:01.906915 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82006b0-e040-4f12-b6fb-b6ac5c2fc6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:19:01.907188 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82006b0-e040-4f12-b6fb-b6ac5c2fc6a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:19:01.914355 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82006b0-e040-4f12-b6fb-b6ac5c2fc6a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:19:01.915847 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82006b0-e040-4f12-b6fb-b6ac5c2fc6a4 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:19:01.930278 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application [None req-b82006b0-e040-4f12-b6fb-b6ac5c2fc6a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:19:01.930278 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:19:01.930278 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:19:01.930278 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:19:01.930278 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:19:01.930278 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:19:01.930278 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:19:01.930278 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:19:01.930278 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:19:01.930278 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:19:01.930278 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:19:01.930278 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:19:01.930278 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 15:19:01.930278 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 15:19:01.930278 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 15:19:01.931507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 15:19:01.931507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:19:01.931507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:19:01.931507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:19:01.931507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:19:01.931507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:19:01.931507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:19:01.931507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:19:01.931507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application with Lock( Jul 07 15:19:01.931507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:19:01.931507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:19:01.931507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:19:01.931507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:19:01.931507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:19:01.931507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:19:01.931507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:19:01.931507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:19:01.932538 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 15:19:01.932538 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 15:19:01.932538 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 15:19:01.932538 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 15:19:01.932538 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 15:19:01.932538 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 15:19:01.932538 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 15:19:01.932538 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: ERROR keystone.server.flask.application Jul 07 15:19:01.932538 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 630/1285] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Jul 7 15:19:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:19:01.937757 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd739a-64c1-47c8-a30b-aaeaae0c136b None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:19:01.988095 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:19:02.036852 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd739a-64c1-47c8-a30b-aaeaae0c136b None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:19:02.070464 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a4f13b-3755-4658-97c7-12b52ef87462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:19:02.071118 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4f13b-3755-4658-97c7-12b52ef87462 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:19:02.071393 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4f13b-3755-4658-97c7-12b52ef87462 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:19:02.071700 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4f13b-3755-4658-97c7-12b52ef87462 None admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:19:02.072472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a4f13b-3755-4658-97c7-12b52ef87462 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:19:02.072765 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a4f13b-3755-4658-97c7-12b52ef87462 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:19:02.074175 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a4f13b-3755-4658-97c7-12b52ef87462 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:19:02.086518 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 656/1286] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Jul 7 15:19:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:19:02.092730 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82006b0-e040-4f12-b6fb-b6ac5c2fc6a4 None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:19:02.201035 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82006b0-e040-4f12-b6fb-b6ac5c2fc6a4 None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:19:02.234736 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471fc6e0-9feb-40c6-abd0-6bbdc0cfdf56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:19:02.235408 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471fc6e0-9feb-40c6-abd0-6bbdc0cfdf56 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:19:02.235723 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471fc6e0-9feb-40c6-abd0-6bbdc0cfdf56 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:19:02.235995 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471fc6e0-9feb-40c6-abd0-6bbdc0cfdf56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:19:02.236816 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471fc6e0-9feb-40c6-abd0-6bbdc0cfdf56 None admin] RBAC: Adding query filter params (role.id=0e7793fbf3064037984da651879e3a94, scope.project.id=c5260c7fd3614e9e93b167c86a94260e, user.id=755c2782cfb042f89950b8fc862e4fa1) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:19:02.237124 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471fc6e0-9feb-40c6-abd0-6bbdc0cfdf56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:19:02.238597 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471fc6e0-9feb-40c6-abd0-6bbdc0cfdf56 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:19:02.244289 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 631/1287] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Jul 7 15:19:02 2022] GET /identity/v3/role_assignments?user.id=755c2782cfb042f89950b8fc862e4fa1&scope.project.id=c5260c7fd3614e9e93b167c86a94260e&role.id=0e7793fbf3064037984da651879e3a94 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:19:04.332521 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e760bfc-4973-4aac-9c7e-5cd55da15b97 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:19:04.333051 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e760bfc-4973-4aac-9c7e-5cd55da15b97 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:19:04.333051 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e760bfc-4973-4aac-9c7e-5cd55da15b97 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:19:04.334451 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 657/1288] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:19:04.339758 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd32de2-cd04-483a-8630-c9648213151b None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:19:04.340159 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd32de2-cd04-483a-8630-c9648213151b None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:19:04.340309 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd32de2-cd04-483a-8630-c9648213151b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:19:04.358392 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-cdd32de2-cd04-483a-8630-c9648213151b None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:19:04.359594 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-cdd32de2-cd04-483a-8630-c9648213151b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:19:04.388830 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:19:04.430360 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 632/1289] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:19:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:19:04.452712 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e760bfc-4973-4aac-9c7e-5cd55da15b97 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:19:04.552758 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e760bfc-4973-4aac-9c7e-5cd55da15b97 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:19:04.562978 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:19:04.586087 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f3c579-374b-4e74-98c1-0b6975c2c573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:19:04.586810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3c579-374b-4e74-98c1-0b6975c2c573 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:19:04.587090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3c579-374b-4e74-98c1-0b6975c2c573 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:19:04.587361 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3c579-374b-4e74-98c1-0b6975c2c573 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:19:04.588038 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f3c579-374b-4e74-98c1-0b6975c2c573 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:19:04.589437 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f3c579-374b-4e74-98c1-0b6975c2c573 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:19:04.601872 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-04f3c579-374b-4e74-98c1-0b6975c2c573 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 07 15:19:04.601872 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:19:04.601872 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 15:19:04.601872 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 15:19:04.601872 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 15:19:04.601872 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 15:19:04.601872 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 15:19:04.601872 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:19:04.601872 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 07 15:19:04.601872 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:19:04.601872 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 15:19:04.601872 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:19:04.601872 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:19:04.601872 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:19:04.601872 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:19:04.601872 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:19:04.601872 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:19:04.603085 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:19:04.603085 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:19:04.603085 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:19:04.603085 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:19:04.603085 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:19:04.603085 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:19:04.603085 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 15:19:04.603085 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 15:19:04.603085 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 15:19:04.603085 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 15:19:04.603085 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:19:04.603085 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:19:04.603085 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:19:04.603085 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:19:04.603085 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:19:04.603085 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:19:04.604005 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:19:04.604005 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:19:04.604005 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:19:04.604005 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:19:04.604005 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:19:04.604005 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:19:04.604005 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:19:04.604005 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:19:04.604005 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:19:04.604005 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:19:04.604005 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 15:19:04.604005 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:19:04.604005 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 07 15:19:04.604005 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:19:04.604005 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 658/1290] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Jul 7 15:19:04 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:19:04.608990 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd32de2-cd04-483a-8630-c9648213151b None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:19:04.707438 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd32de2-cd04-483a-8630-c9648213151b None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:19:04.742592 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0130d2d3-b5bb-4c1d-899f-409273289a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:19:04.743248 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130d2d3-b5bb-4c1d-899f-409273289a84 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:19:04.743523 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130d2d3-b5bb-4c1d-899f-409273289a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:19:04.743824 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0130d2d3-b5bb-4c1d-899f-409273289a84 None admin] PATH_INFO: `/v3/services` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:19:04.744520 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0130d2d3-b5bb-4c1d-899f-409273289a84 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:19:04.744977 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0130d2d3-b5bb-4c1d-899f-409273289a84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:19:04.746232 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0130d2d3-b5bb-4c1d-899f-409273289a84 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:19:04.750987 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 633/1291] 10.222.0.25 () {56 vars in 1116 bytes} [Thu Jul 7 15:19:04 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:19:04.756617 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f3c579-374b-4e74-98c1-0b6975c2c573 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:19:04.860984 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f3c579-374b-4e74-98c1-0b6975c2c573 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:19:04.893693 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a2a79-70fc-4ccf-8765-673eb44c6ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:19:04.894349 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a2a79-70fc-4ccf-8765-673eb44c6ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:19:04.894632 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a2a79-70fc-4ccf-8765-673eb44c6ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:19:04.894908 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a2a79-70fc-4ccf-8765-673eb44c6ef7 None admin] PATH_INFO: `/v3/services` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:19:04.895630 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a2a79-70fc-4ccf-8765-673eb44c6ef7 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:19:04.895938 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a2a79-70fc-4ccf-8765-673eb44c6ef7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:19:04.897400 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a2a79-70fc-4ccf-8765-673eb44c6ef7 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:19:04.902300 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 659/1292] 10.222.0.25 () {56 vars in 1116 bytes} [Thu Jul 7 15:19:04 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:19:06.910066 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9e048-5faa-4f9d-a735-0eece8d69f0e None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:19:06.910641 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9e048-5faa-4f9d-a735-0eece8d69f0e None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:19:06.910641 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9e048-5faa-4f9d-a735-0eece8d69f0e None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:19:06.912023 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 634/1293] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:19:06.917863 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13869f-8810-4ff0-8ac5-a55884eae53d None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:19:06.918283 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13869f-8810-4ff0-8ac5-a55884eae53d None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:19:06.918433 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13869f-8810-4ff0-8ac5-a55884eae53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:19:06.934936 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-3c13869f-8810-4ff0-8ac5-a55884eae53d None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:19:06.936136 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-3c13869f-8810-4ff0-8ac5-a55884eae53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:19:06.966702 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:19:07.002990 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 660/1294] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:19:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:19:07.027024 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a9e048-5faa-4f9d-a735-0eece8d69f0e None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:19:07.057670 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:19:07.126021 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a9e048-5faa-4f9d-a735-0eece8d69f0e None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:19:07.159664 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1706ac2-0d60-4cf8-a5fc-6fa1b0fa2fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:19:07.160359 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1706ac2-0d60-4cf8-a5fc-6fa1b0fa2fb2 None admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:19:07.160591 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1706ac2-0d60-4cf8-a5fc-6fa1b0fa2fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:19:07.160863 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1706ac2-0d60-4cf8-a5fc-6fa1b0fa2fb2 None admin] PATH_INFO: `/v3/services` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:19:07.161577 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1706ac2-0d60-4cf8-a5fc-6fa1b0fa2fb2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:19:07.162910 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1706ac2-0d60-4cf8-a5fc-6fa1b0fa2fb2 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:19:07.253354 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 635/1295] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Jul 7 15:19:07 2022] POST /identity/v3/services => generated 246 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:19:09.248301 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f9008-cfa4-468e-8427-0a7f862c19ac None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:19:09.248816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f9008-cfa4-468e-8427-0a7f862c19ac None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:19:09.248816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f9008-cfa4-468e-8427-0a7f862c19ac None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:19:09.250199 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 661/1296] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:19:09.255213 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d0dc2-bd5f-44a3-ac0b-c18438917113 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:19:09.255646 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d0dc2-bd5f-44a3-ac0b-c18438917113 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:19:09.255802 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d0dc2-bd5f-44a3-ac0b-c18438917113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:19:09.272861 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-d12d0dc2-bd5f-44a3-ac0b-c18438917113 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:19:09.274043 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-d12d0dc2-bd5f-44a3-ac0b-c18438917113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:19:09.345216 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 636/1297] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:19:09 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:19:09.368965 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f9008-cfa4-468e-8427-0a7f862c19ac None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:19:09.467707 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f9008-cfa4-468e-8427-0a7f862c19ac None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:19:09.500676 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfd4ade-89bd-4f74-8ef6-cc59571de0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:19:09.501395 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd4ade-89bd-4f74-8ef6-cc59571de0d3 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:19:09.501672 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd4ade-89bd-4f74-8ef6-cc59571de0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:19:09.501943 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd4ade-89bd-4f74-8ef6-cc59571de0d3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:19:09.502579 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfd4ade-89bd-4f74-8ef6-cc59571de0d3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:19:09.503962 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfd4ade-89bd-4f74-8ef6-cc59571de0d3 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:19:09.515425 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-2dfd4ade-89bd-4f74-8ef6-cc59571de0d3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 07 15:19:09.515425 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:19:09.515425 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 15:19:09.515425 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 15:19:09.515425 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 15:19:09.515425 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 15:19:09.515425 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 15:19:09.515425 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:19:09.515425 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 07 15:19:09.515425 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:19:09.515425 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 15:19:09.515425 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:19:09.515425 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:19:09.515425 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:19:09.515425 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:19:09.515425 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:19:09.515425 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:19:09.516647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:19:09.516647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:19:09.516647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:19:09.516647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:19:09.516647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:19:09.516647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:19:09.516647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 15:19:09.516647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 15:19:09.516647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 15:19:09.516647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 15:19:09.516647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:19:09.516647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:19:09.516647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:19:09.516647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:19:09.516647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:19:09.516647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:19:09.517677 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:19:09.517677 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:19:09.517677 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:19:09.517677 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:19:09.517677 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:19:09.517677 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:19:09.517677 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:19:09.517677 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:19:09.517677 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:19:09.517677 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:19:09.517677 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 15:19:09.517677 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:19:09.517677 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 07 15:19:09.517677 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:19:09.517677 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 662/1298] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Jul 7 15:19:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:19:09.522581 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d0dc2-bd5f-44a3-ac0b-c18438917113 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:19:09.541060 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:19:09.619336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d0dc2-bd5f-44a3-ac0b-c18438917113 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:19:09.653070 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee9d40a-055b-4c41-80e2-eba3446a96da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:19:09.653715 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9d40a-055b-4c41-80e2-eba3446a96da None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:19:09.653998 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9d40a-055b-4c41-80e2-eba3446a96da None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:19:09.654274 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9d40a-055b-4c41-80e2-eba3446a96da None admin] PATH_INFO: `/v3/services` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:19:09.654971 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee9d40a-055b-4c41-80e2-eba3446a96da None admin] RBAC: Adding query filter params (name=placement) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:19:09.655280 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee9d40a-055b-4c41-80e2-eba3446a96da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:19:09.656727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee9d40a-055b-4c41-80e2-eba3446a96da None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:19:09.661770 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 637/1299] 10.222.0.25 () {56 vars in 1116 bytes} [Thu Jul 7 15:19:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:19:09.668333 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfd4ade-89bd-4f74-8ef6-cc59571de0d3 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:19:09.698999 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:19:09.763751 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfd4ade-89bd-4f74-8ef6-cc59571de0d3 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:19:09.797671 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d06d10-1b33-4d99-980b-63d3acb82cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:19:09.798314 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d06d10-1b33-4d99-980b-63d3acb82cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:19:09.798593 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d06d10-1b33-4d99-980b-63d3acb82cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:19:09.798864 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d06d10-1b33-4d99-980b-63d3acb82cf3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:19:09.799666 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d06d10-1b33-4d99-980b-63d3acb82cf3 None admin] RBAC: Adding query filter params (interface=public, service_id=9c168c2b9866450a919fc87f8caf7f18, region_id=RegionOne) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:19:09.799991 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d06d10-1b33-4d99-980b-63d3acb82cf3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:19:09.801327 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d06d10-1b33-4d99-980b-63d3acb82cf3 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:19:09.806562 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 663/1300] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Jul 7 15:19:09 2022] GET /identity/v3/endpoints?service_id=9c168c2b9866450a919fc87f8caf7f18&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:19:09.811828 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee9d40a-055b-4c41-80e2-eba3446a96da None admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:19:09.891249 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:19:09.909271 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee9d40a-055b-4c41-80e2-eba3446a96da None admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:19:09.942936 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461b7dfb-a59d-471d-bfea-db745a33ef42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:19:09.943596 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b7dfb-a59d-471d-bfea-db745a33ef42 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:19:09.943878 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b7dfb-a59d-471d-bfea-db745a33ef42 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:19:09.944176 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b7dfb-a59d-471d-bfea-db745a33ef42 None admin] PATH_INFO: `/v3/services` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:19:09.944844 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461b7dfb-a59d-471d-bfea-db745a33ef42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:19:09.946232 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461b7dfb-a59d-471d-bfea-db745a33ef42 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:19:09.953200 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 638/1301] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Jul 7 15:19:09 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:19:11.942331 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a8948-b4ce-4428-ba02-abf595db85f9 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:19:11.942877 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a8948-b4ce-4428-ba02-abf595db85f9 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:19:11.942877 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a8948-b4ce-4428-ba02-abf595db85f9 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:19:11.944318 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 664/1302] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:19:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:19:11.949540 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95124c01-ed01-4463-9083-db20adbc90c2 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:19:11.949939 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95124c01-ed01-4463-9083-db20adbc90c2 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:19:11.950120 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95124c01-ed01-4463-9083-db20adbc90c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:19:11.967452 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-95124c01-ed01-4463-9083-db20adbc90c2 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:19:11.968660 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-95124c01-ed01-4463-9083-db20adbc90c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:19:12.035315 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 639/1303] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:19:11 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 15:19:12.058475 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8a8948-b4ce-4428-ba02-abf595db85f9 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:19:12.156004 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:19:12.162926 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8a8948-b4ce-4428-ba02-abf595db85f9 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:19:12.197493 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5258dc1a-2aaa-4016-9408-3f7e743b0ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:19:12.198212 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258dc1a-2aaa-4016-9408-3f7e743b0ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:19:12.198494 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258dc1a-2aaa-4016-9408-3f7e743b0ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:19:12.198772 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258dc1a-2aaa-4016-9408-3f7e743b0ff1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:19:12.199408 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5258dc1a-2aaa-4016-9408-3f7e743b0ff1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:19:12.200870 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5258dc1a-2aaa-4016-9408-3f7e743b0ff1 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:19:12.213190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application [None req-5258dc1a-2aaa-4016-9408-3f7e743b0ff1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 07 15:19:12.213190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:19:12.213190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 15:19:12.213190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 15:19:12.213190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 15:19:12.213190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 15:19:12.213190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 15:19:12.213190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:19:12.213190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 07 15:19:12.213190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:19:12.213190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 15:19:12.213190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:19:12.213190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 15:19:12.213190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 15:19:12.213190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 15:19:12.213190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 15:19:12.213190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 15:19:12.219139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 15:19:12.219139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 15:19:12.219139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 15:19:12.219139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 15:19:12.219139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 15:19:12.219139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 15:19:12.219139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 15:19:12.219139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 15:19:12.219139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 15:19:12.219139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 15:19:12.219139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 15:19:12.219139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 15:19:12.219139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 15:19:12.219139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 15:19:12.219139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 15:19:12.219139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 15:19:12.220316 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 15:19:12.220316 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application with Lock( Jul 07 15:19:12.220316 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 15:19:12.220316 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self._enter() Jul 07 15:19:12.220316 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 15:19:12.220316 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 15:19:12.220316 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 15:19:12.220316 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application return self.creator() Jul 07 15:19:12.220316 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 15:19:12.220316 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application created_value = creator( Jul 07 15:19:12.220316 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 15:19:12.220316 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 15:19:12.220316 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 07 15:19:12.220316 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: ERROR keystone.server.flask.application Jul 07 15:19:12.220316 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 665/1304] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Jul 7 15:19:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 15:19:12.221573 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95124c01-ed01-4463-9083-db20adbc90c2 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:19:12.294977 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:19:12.316464 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95124c01-ed01-4463-9083-db20adbc90c2 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:19:12.349943 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26dfa66-4434-4511-a124-da094701c466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:19:12.350590 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26dfa66-4434-4511-a124-da094701c466 None admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:19:12.350871 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26dfa66-4434-4511-a124-da094701c466 None admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:19:12.351145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26dfa66-4434-4511-a124-da094701c466 None admin] PATH_INFO: `/v3/services` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:19:12.351878 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26dfa66-4434-4511-a124-da094701c466 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:19:12.352186 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26dfa66-4434-4511-a124-da094701c466 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:19:12.353539 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26dfa66-4434-4511-a124-da094701c466 None admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:19:12.359414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 640/1305] 10.222.0.25 () {56 vars in 1116 bytes} [Thu Jul 7 15:19:12 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:19:12.365037 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5258dc1a-2aaa-4016-9408-3f7e743b0ff1 None admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:19:12.462963 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5258dc1a-2aaa-4016-9408-3f7e743b0ff1 None admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:19:12.494762 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7327c-058e-4f6f-8d52-5a11f06b6c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:19:12.495391 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7327c-058e-4f6f-8d52-5a11f06b6c19 None admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:19:12.495697 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7327c-058e-4f6f-8d52-5a11f06b6c19 None admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:19:12.495969 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7327c-058e-4f6f-8d52-5a11f06b6c19 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:19:12.496685 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7327c-058e-4f6f-8d52-5a11f06b6c19 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:19:12.498037 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7327c-058e-4f6f-8d52-5a11f06b6c19 None admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:19:12.578851 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 666/1306] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Jul 7 15:19:12 2022] POST /identity/v3/endpoints => generated 331 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:19:22.356817 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=30 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:19:22.574424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:24:53.666381 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70694374-ff49-4786-9806-acd5923941f7 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:24:53.666969 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70694374-ff49-4786-9806-acd5923941f7 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:24:53.666969 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70694374-ff49-4786-9806-acd5923941f7 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:24:53.668512 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 641/1307] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:24:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:24:53.674326 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12121a4d-7ba5-4ea5-87f2-a5a59a2cef1d None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:24:53.674790 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12121a4d-7ba5-4ea5-87f2-a5a59a2cef1d None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:24:53.674874 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12121a4d-7ba5-4ea5-87f2-a5a59a2cef1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:24:53.760038 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-12121a4d-7ba5-4ea5-87f2-a5a59a2cef1d None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:24:53.761336 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-12121a4d-7ba5-4ea5-87f2-a5a59a2cef1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:24:53.916321 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 667/1308] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:24:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:24:53.968831 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1ac8d-7b5c-4626-b007-3e500ec0ab47 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:24:53.969396 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1ac8d-7b5c-4626-b007-3e500ec0ab47 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:24:53.969866 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1ac8d-7b5c-4626-b007-3e500ec0ab47 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:24:53.972338 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 642/1309] 10.222.0.25 () {54 vars in 964 bytes} [Thu Jul 7 15:24:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:24:53.978585 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cef9a3-ba8a-4ce1-9d63-d257ea5a5ce8 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:24:53.978585 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cef9a3-ba8a-4ce1-9d63-d257ea5a5ce8 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:24:53.978809 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cef9a3-ba8a-4ce1-9d63-d257ea5a5ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:24:54.076187 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-50cef9a3-ba8a-4ce1-9d63-d257ea5a5ce8 None None] MFA Rules not processed for user `e79efaea155c46ecb9831956e68aec8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:24:54.077924 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-50cef9a3-ba8a-4ce1-9d63-d257ea5a5ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:24:54.102704 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:24:54.165282 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 668/1310] 10.222.0.25 () {58 vars in 1061 bytes} [Thu Jul 7 15:24:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:24:54.175063 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b1ac8d-7b5c-4626-b007-3e500ec0ab47 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:24:54.176826 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-04b1ac8d-7b5c-4626-b007-3e500ec0ab47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:24:54.299549 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b1ac8d-7b5c-4626-b007-3e500ec0ab47 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:24:54.376637 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2d1a0-a100-41a8-9b89-5ce4d2a2a58e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:24:54.377286 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2d1a0-a100-41a8-9b89-5ce4d2a2a58e service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:24:54.377569 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2d1a0-a100-41a8-9b89-5ce4d2a2a58e service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:24:54.377845 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2d1a0-a100-41a8-9b89-5ce4d2a2a58e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:24:54.379475 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-77d2d1a0-a100-41a8-9b89-5ce4d2a2a58e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:24:54.436858 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:24:54.492490 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d2d1a0-a100-41a8-9b89-5ce4d2a2a58e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:24:54.493940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d2d1a0-a100-41a8-9b89-5ce4d2a2a58e service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:24:54.514592 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 643/1311] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:24:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:24:56.843609 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95f0d1-aedc-4f51-a4e3-b2446bfa5296 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:24:56.844734 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95f0d1-aedc-4f51-a4e3-b2446bfa5296 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:24:56.844734 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95f0d1-aedc-4f51-a4e3-b2446bfa5296 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:24:56.846032 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 669/1312] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:24:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:24:56.851427 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4468fbc-be0d-44d9-b0a3-9baa6b4ab89e None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:24:56.851903 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4468fbc-be0d-44d9-b0a3-9baa6b4ab89e None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:24:56.852004 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4468fbc-be0d-44d9-b0a3-9baa6b4ab89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:24:56.932834 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-f4468fbc-be0d-44d9-b0a3-9baa6b4ab89e None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:24:56.934057 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-f4468fbc-be0d-44d9-b0a3-9baa6b4ab89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:24:57.039737 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 644/1313] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:24:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:24:57.093096 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66115ad9-051b-425e-87a2-3c06c19e79fa None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:24:57.093409 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66115ad9-051b-425e-87a2-3c06c19e79fa None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:24:57.093736 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66115ad9-051b-425e-87a2-3c06c19e79fa None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:24:57.095600 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 670/1314] 10.222.0.25 () {54 vars in 964 bytes} [Thu Jul 7 15:24:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:24:57.101757 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8aa9a-3ecd-4d11-a71b-0b251f743785 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:24:57.102048 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8aa9a-3ecd-4d11-a71b-0b251f743785 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:24:57.102327 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8aa9a-3ecd-4d11-a71b-0b251f743785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:24:57.113621 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:24:57.177145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-31d8aa9a-3ecd-4d11-a71b-0b251f743785 None None] MFA Rules not processed for user `e79efaea155c46ecb9831956e68aec8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:24:57.178367 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-31d8aa9a-3ecd-4d11-a71b-0b251f743785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:24:57.256839 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 645/1315] 10.222.0.25 () {58 vars in 1061 bytes} [Thu Jul 7 15:24:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:24:57.265617 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66115ad9-051b-425e-87a2-3c06c19e79fa None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:24:57.267690 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-66115ad9-051b-425e-87a2-3c06c19e79fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:24:57.371952 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66115ad9-051b-425e-87a2-3c06c19e79fa None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:24:57.442972 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:24:57.445539 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e9be7-982f-41b5-b151-ba80809601dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:24:57.446183 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e9be7-982f-41b5-b151-ba80809601dc service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:24:57.446470 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e9be7-982f-41b5-b151-ba80809601dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:24:57.446748 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e9be7-982f-41b5-b151-ba80809601dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:24:57.448430 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-e48e9be7-982f-41b5-b151-ba80809601dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:24:57.556263 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48e9be7-982f-41b5-b151-ba80809601dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:24:57.557686 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48e9be7-982f-41b5-b151-ba80809601dc service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:24:57.577829 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 671/1316] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:24:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:25:00.294336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa7497-0b30-4818-a110-1795c16aa82a None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:00.294858 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa7497-0b30-4818-a110-1795c16aa82a None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:00.294858 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa7497-0b30-4818-a110-1795c16aa82a None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:00.296230 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 646/1317] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:25:00.301868 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980bc4ff-5fa6-41df-a8db-26a5db05a3d2 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:00.302269 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980bc4ff-5fa6-41df-a8db-26a5db05a3d2 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:00.302422 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980bc4ff-5fa6-41df-a8db-26a5db05a3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:00.365346 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-980bc4ff-5fa6-41df-a8db-26a5db05a3d2 None None] MFA Rules not processed for user `c4236ad3d5d949cf9d36da8484c8bf39`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:25:00.366537 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-980bc4ff-5fa6-41df-a8db-26a5db05a3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:00.416360 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:25:00.460315 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 672/1318] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:25:00 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:25:00.510454 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa7497-0b30-4818-a110-1795c16aa82a None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:25:00.512057 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-88fa7497-0b30-4818-a110-1795c16aa82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:00.616970 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa7497-0b30-4818-a110-1795c16aa82a None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:25:00.619805 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:25:00.692535 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b98480-9d42-440f-9c8d-c0da7043ab7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:25:00.693205 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b98480-9d42-440f-9c8d-c0da7043ab7f service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:00.693486 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b98480-9d42-440f-9c8d-c0da7043ab7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:00.693756 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b98480-9d42-440f-9c8d-c0da7043ab7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:00.695400 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-07b98480-9d42-440f-9c8d-c0da7043ab7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:00.803298 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b98480-9d42-440f-9c8d-c0da7043ab7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:25:00.804771 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b98480-9d42-440f-9c8d-c0da7043ab7f service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:25:00.824438 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 647/1319] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:25:00 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:25:03.651439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a84fe3-db8f-4061-a10a-d1d852f25f43 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:03.651977 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a84fe3-db8f-4061-a10a-d1d852f25f43 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:03.651977 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a84fe3-db8f-4061-a10a-d1d852f25f43 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:03.653300 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 673/1320] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:25:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:25:03.659053 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f061f-74ec-4b20-8ff4-696869b4b23a None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:03.659472 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f061f-74ec-4b20-8ff4-696869b4b23a None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:03.659630 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f061f-74ec-4b20-8ff4-696869b4b23a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:03.698954 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:25:03.723342 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-6b4f061f-74ec-4b20-8ff4-696869b4b23a None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:25:03.724563 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-6b4f061f-74ec-4b20-8ff4-696869b4b23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:03.824844 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 648/1321] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:25:03 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:25:03.863476 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a84fe3-db8f-4061-a10a-d1d852f25f43 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:25:03.865017 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-c2a84fe3-db8f-4061-a10a-d1d852f25f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:03.965697 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a84fe3-db8f-4061-a10a-d1d852f25f43 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:25:04.038851 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b08e4c-7f02-4e56-a376-3d2b2e88d4d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:25:04.039491 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b08e4c-7f02-4e56-a376-3d2b2e88d4d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:04.039810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b08e4c-7f02-4e56-a376-3d2b2e88d4d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:04.040087 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b08e4c-7f02-4e56-a376-3d2b2e88d4d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:04.041725 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-87b08e4c-7f02-4e56-a376-3d2b2e88d4d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:04.049116 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:25:04.148761 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b08e4c-7f02-4e56-a376-3d2b2e88d4d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:25:04.150188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b08e4c-7f02-4e56-a376-3d2b2e88d4d1 service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:25:04.168529 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 674/1322] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:25:03 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:25:07.284811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b9567-73fc-4d64-a95a-50b2879ef836 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:07.285334 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b9567-73fc-4d64-a95a-50b2879ef836 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:07.285334 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b9567-73fc-4d64-a95a-50b2879ef836 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:07.286809 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 649/1323] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:25:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:25:07.291983 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156fdec9-0a3f-4b5b-94de-9716a99d10c1 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:07.292430 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156fdec9-0a3f-4b5b-94de-9716a99d10c1 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:07.292535 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156fdec9-0a3f-4b5b-94de-9716a99d10c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:07.359286 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-156fdec9-0a3f-4b5b-94de-9716a99d10c1 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:25:07.360619 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-156fdec9-0a3f-4b5b-94de-9716a99d10c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:07.422490 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:25:07.465336 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 675/1324] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:25:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:25:07.502053 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440b9567-73fc-4d64-a95a-50b2879ef836 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:25:07.503628 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-440b9567-73fc-4d64-a95a-50b2879ef836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:07.615078 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440b9567-73fc-4d64-a95a-50b2879ef836 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:25:07.627162 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:25:07.688575 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f990f9-81e8-4469-8fbc-40edeb6e5dd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:25:07.689749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f990f9-81e8-4469-8fbc-40edeb6e5dd4 service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:07.690037 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f990f9-81e8-4469-8fbc-40edeb6e5dd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:07.690306 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f990f9-81e8-4469-8fbc-40edeb6e5dd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:07.691987 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-31f990f9-81e8-4469-8fbc-40edeb6e5dd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:07.807403 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f990f9-81e8-4469-8fbc-40edeb6e5dd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:25:07.808746 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f990f9-81e8-4469-8fbc-40edeb6e5dd4 service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:25:07.826940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 650/1325] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:25:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:25:09.930409 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de913da-27be-45fe-ae37-80da6d67cba0 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:09.930916 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de913da-27be-45fe-ae37-80da6d67cba0 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:09.930916 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de913da-27be-45fe-ae37-80da6d67cba0 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:09.932236 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 676/1326] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:25:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:25:09.937363 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab2a44-d8d1-4c5c-9596-cba71e8c6ee1 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:09.937768 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab2a44-d8d1-4c5c-9596-cba71e8c6ee1 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:09.938045 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab2a44-d8d1-4c5c-9596-cba71e8c6ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:09.987499 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:25:10.005373 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-daab2a44-d8d1-4c5c-9596-cba71e8c6ee1 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:25:10.006602 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-daab2a44-d8d1-4c5c-9596-cba71e8c6ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:10.101863 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 651/1327] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:25:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:25:10.138271 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de913da-27be-45fe-ae37-80da6d67cba0 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:25:10.139950 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-8de913da-27be-45fe-ae37-80da6d67cba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:10.244442 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de913da-27be-45fe-ae37-80da6d67cba0 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:25:10.322292 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434eda8c-5bf3-4810-8dcd-fb02b9d571e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:25:10.322964 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434eda8c-5bf3-4810-8dcd-fb02b9d571e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:10.323236 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434eda8c-5bf3-4810-8dcd-fb02b9d571e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:10.323504 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434eda8c-5bf3-4810-8dcd-fb02b9d571e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:10.325154 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-434eda8c-5bf3-4810-8dcd-fb02b9d571e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:10.342643 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:25:10.434893 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434eda8c-5bf3-4810-8dcd-fb02b9d571e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:25:10.436473 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434eda8c-5bf3-4810-8dcd-fb02b9d571e6 service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:25:10.480098 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 677/1328] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:25:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:25:12.589482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f34a9-07e2-43c4-9686-611d7601cd0f None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:12.589934 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f34a9-07e2-43c4-9686-611d7601cd0f None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:12.590072 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f34a9-07e2-43c4-9686-611d7601cd0f None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:12.591495 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 652/1329] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:25:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:25:12.597733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfb50c-b39a-422b-910d-9bce9764ba53 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:12.598156 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfb50c-b39a-422b-910d-9bce9764ba53 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:12.598156 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfb50c-b39a-422b-910d-9bce9764ba53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:12.665603 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-dfdfb50c-b39a-422b-910d-9bce9764ba53 None None] MFA Rules not processed for user `c4236ad3d5d949cf9d36da8484c8bf39`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:25:12.666854 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-dfdfb50c-b39a-422b-910d-9bce9764ba53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:12.733472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:25:12.767245 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 678/1330] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:25:12 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:25:12.804496 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f34a9-07e2-43c4-9686-611d7601cd0f None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:25:12.806011 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-1d1f34a9-07e2-43c4-9686-611d7601cd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:12.915918 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f34a9-07e2-43c4-9686-611d7601cd0f None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:25:12.935317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:25:12.994174 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530efa9-53b8-43da-bd18-41daba3f4125 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:25:12.994831 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530efa9-53b8-43da-bd18-41daba3f4125 service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:12.995109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530efa9-53b8-43da-bd18-41daba3f4125 service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:12.995385 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530efa9-53b8-43da-bd18-41daba3f4125 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:12.997083 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-4530efa9-53b8-43da-bd18-41daba3f4125 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:13.119216 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4530efa9-53b8-43da-bd18-41daba3f4125 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:25:13.120754 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4530efa9-53b8-43da-bd18-41daba3f4125 service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:25:13.140126 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 653/1331] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:25:12 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:25:15.590591 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba8ad2-5400-4cc3-90e4-772f1140b189 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:15.591155 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba8ad2-5400-4cc3-90e4-772f1140b189 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:15.591155 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba8ad2-5400-4cc3-90e4-772f1140b189 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:15.592566 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 679/1332] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:25:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:25:15.597815 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9f7a6-d95b-45e9-a0e2-81dd9ec2447e None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:15.598197 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9f7a6-d95b-45e9-a0e2-81dd9ec2447e None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:15.598370 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9f7a6-d95b-45e9-a0e2-81dd9ec2447e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:15.664150 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:25:15.666846 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-76a9f7a6-d95b-45e9-a0e2-81dd9ec2447e None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:25:15.668096 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-76a9f7a6-d95b-45e9-a0e2-81dd9ec2447e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:15.762926 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 654/1333] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:25:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:25:15.797460 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba8ad2-5400-4cc3-90e4-772f1140b189 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:25:15.799651 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-0eba8ad2-5400-4cc3-90e4-772f1140b189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:15.911714 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba8ad2-5400-4cc3-90e4-772f1140b189 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:25:15.986351 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab1fb4-7960-49a1-bcea-ea226cee9178 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:25:15.986999 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab1fb4-7960-49a1-bcea-ea226cee9178 service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:15.987281 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab1fb4-7960-49a1-bcea-ea226cee9178 service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:15.987556 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab1fb4-7960-49a1-bcea-ea226cee9178 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:15.989211 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-b9ab1fb4-7960-49a1-bcea-ea226cee9178 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:16.015669 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:25:16.095914 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab1fb4-7960-49a1-bcea-ea226cee9178 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:25:16.097365 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab1fb4-7960-49a1-bcea-ea226cee9178 service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:25:16.115484 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 680/1334] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:25:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:25:18.190453 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564fbe7a-26bf-4353-8bc3-275af897933c None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:18.191008 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564fbe7a-26bf-4353-8bc3-275af897933c None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:18.191008 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564fbe7a-26bf-4353-8bc3-275af897933c None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:18.192409 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 655/1335] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:25:18.197419 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035cdf87-ef2d-411f-a95f-4478858d8b08 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:18.197814 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035cdf87-ef2d-411f-a95f-4478858d8b08 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:18.197966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035cdf87-ef2d-411f-a95f-4478858d8b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:18.261988 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-035cdf87-ef2d-411f-a95f-4478858d8b08 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:25:18.263100 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-035cdf87-ef2d-411f-a95f-4478858d8b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:18.336365 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:25:18.357842 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 681/1336] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:25:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:25:18.398102 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564fbe7a-26bf-4353-8bc3-275af897933c None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:25:18.399640 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-564fbe7a-26bf-4353-8bc3-275af897933c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:18.500202 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564fbe7a-26bf-4353-8bc3-275af897933c None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:25:18.528621 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:25:18.573835 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b1b99c-2d38-4a06-873e-b44304415721 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:25:18.574472 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1b99c-2d38-4a06-873e-b44304415721 service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:18.574754 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1b99c-2d38-4a06-873e-b44304415721 service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:18.575028 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1b99c-2d38-4a06-873e-b44304415721 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:18.576686 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-14b1b99c-2d38-4a06-873e-b44304415721 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:18.683880 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b1b99c-2d38-4a06-873e-b44304415721 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:25:18.685291 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b1b99c-2d38-4a06-873e-b44304415721 service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:25:18.704088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 656/1337] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:25:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:25:21.280225 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777e0c0-def2-4d0a-9f4a-9fd0a6efbc57 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:21.280748 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777e0c0-def2-4d0a-9f4a-9fd0a6efbc57 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:21.280748 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777e0c0-def2-4d0a-9f4a-9fd0a6efbc57 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:21.282099 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 682/1338] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:25:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:25:21.287289 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc259d3-cc0a-47bd-a0c8-5c0d0b26e4a5 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:21.287703 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc259d3-cc0a-47bd-a0c8-5c0d0b26e4a5 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:21.287871 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc259d3-cc0a-47bd-a0c8-5c0d0b26e4a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:21.351232 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-7dc259d3-cc0a-47bd-a0c8-5c0d0b26e4a5 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:25:21.352480 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-7dc259d3-cc0a-47bd-a0c8-5c0d0b26e4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:21.361277 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:25:21.448462 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 657/1339] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:25:21 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:25:21.484501 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777e0c0-def2-4d0a-9f4a-9fd0a6efbc57 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:25:21.486024 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-8777e0c0-def2-4d0a-9f4a-9fd0a6efbc57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:21.587916 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777e0c0-def2-4d0a-9f4a-9fd0a6efbc57 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:25:21.658552 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caf8d43-a60a-4f2e-a025-ada14563604a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:25:21.659185 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf8d43-a60a-4f2e-a025-ada14563604a service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:21.659467 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf8d43-a60a-4f2e-a025-ada14563604a service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:21.659764 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf8d43-a60a-4f2e-a025-ada14563604a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:21.661382 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-9caf8d43-a60a-4f2e-a025-ada14563604a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:21.698061 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:25:21.769688 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caf8d43-a60a-4f2e-a025-ada14563604a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:25:21.771117 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caf8d43-a60a-4f2e-a025-ada14563604a service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:25:21.788956 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 683/1340] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:25:21 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:25:25.930378 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249708ad-18d8-408c-845a-bbf4a212e0a5 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:25.930855 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249708ad-18d8-408c-845a-bbf4a212e0a5 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:25.930999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249708ad-18d8-408c-845a-bbf4a212e0a5 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:25.932478 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 658/1341] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:25:25.937686 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b354840-36d5-4ec0-8389-695ca7d1debc None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:25.938138 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b354840-36d5-4ec0-8389-695ca7d1debc None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:25.938269 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b354840-36d5-4ec0-8389-695ca7d1debc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:26.014609 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-6b354840-36d5-4ec0-8389-695ca7d1debc None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:25:26.015839 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-6b354840-36d5-4ec0-8389-695ca7d1debc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:26.111519 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:25:26.124243 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 684/1342] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:25:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:25:26.163257 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249708ad-18d8-408c-845a-bbf4a212e0a5 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:25:26.165535 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-249708ad-18d8-408c-845a-bbf4a212e0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:26.258925 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249708ad-18d8-408c-845a-bbf4a212e0a5 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:25:26.262436 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379c6c18-8be7-4042-ae59-0babccef5a37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:25:26.263071 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c6c18-8be7-4042-ae59-0babccef5a37 service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:26.263348 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c6c18-8be7-4042-ae59-0babccef5a37 service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:26.263657 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c6c18-8be7-4042-ae59-0babccef5a37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:26.273033 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379c6c18-8be7-4042-ae59-0babccef5a37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:25:26.274433 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379c6c18-8be7-4042-ae59-0babccef5a37 service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:25:26.284235 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 659/1343] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:25:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:25:29.361305 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5516720-73f0-4768-b2c8-b3db1904affd None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:29.361738 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5516720-73f0-4768-b2c8-b3db1904affd None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:29.361864 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5516720-73f0-4768-b2c8-b3db1904affd None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:29.363387 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 685/1344] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:25:29.368524 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4f0d8-bd4b-48a1-9495-6913ab9c3955 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:29.368936 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4f0d8-bd4b-48a1-9495-6913ab9c3955 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:29.369210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4f0d8-bd4b-48a1-9495-6913ab9c3955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:29.396925 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-caf4f0d8-bd4b-48a1-9495-6913ab9c3955 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:25:29.398127 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-caf4f0d8-bd4b-48a1-9495-6913ab9c3955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:29.426461 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:25:29.475867 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 660/1345] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:25:29 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:25:29.513312 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5516720-73f0-4768-b2c8-b3db1904affd None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:25:29.515121 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-d5516720-73f0-4768-b2c8-b3db1904affd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:29.529456 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5516720-73f0-4768-b2c8-b3db1904affd None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:25:29.533145 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b974e43e-da6a-4828-926e-91acc66963d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:25:29.533811 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b974e43e-da6a-4828-926e-91acc66963d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:29.534095 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b974e43e-da6a-4828-926e-91acc66963d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:29.534375 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b974e43e-da6a-4828-926e-91acc66963d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:29.546251 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b974e43e-da6a-4828-926e-91acc66963d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:25:29.547762 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b974e43e-da6a-4828-926e-91acc66963d6 service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:25:29.563305 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 686/1346] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:25:29 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:25:34.319324 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b96089-920e-4312-97dd-505d0d4b1d92 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:34.319879 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b96089-920e-4312-97dd-505d0d4b1d92 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:34.319879 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b96089-920e-4312-97dd-505d0d4b1d92 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:34.321266 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 661/1347] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:25:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:25:34.326602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ff64b-9c5a-4b17-b92d-478bd6166f19 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:34.327089 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ff64b-9c5a-4b17-b92d-478bd6166f19 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:34.327198 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ff64b-9c5a-4b17-b92d-478bd6166f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:34.346378 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-dc3ff64b-9c5a-4b17-b92d-478bd6166f19 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:25:34.347498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-dc3ff64b-9c5a-4b17-b92d-478bd6166f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:34.355650 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 687/1348] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:25:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:25:34.393243 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b96089-920e-4312-97dd-505d0d4b1d92 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:25:34.403746 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b96089-920e-4312-97dd-505d0d4b1d92 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:25:34.407969 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c587f-508c-400c-bae8-506631df99ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:25:34.408708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c587f-508c-400c-bae8-506631df99ee service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:34.409054 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c587f-508c-400c-bae8-506631df99ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:34.409397 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c587f-508c-400c-bae8-506631df99ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:34.419291 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597c587f-508c-400c-bae8-506631df99ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:25:34.420846 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597c587f-508c-400c-bae8-506631df99ee service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:25:34.430971 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 662/1349] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:25:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:25:39.311471 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747af74-4649-4ce7-8ecb-62d1ec29eac7 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:39.312941 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747af74-4649-4ce7-8ecb-62d1ec29eac7 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:39.313448 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747af74-4649-4ce7-8ecb-62d1ec29eac7 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:39.315424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 688/1350] 10.222.0.25 () {54 vars in 921 bytes} [Thu Jul 7 15:25:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:25:39.321831 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbed5c2-cc8f-4300-bcc3-d139deafe54e None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:39.324814 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbed5c2-cc8f-4300-bcc3-d139deafe54e None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:39.324814 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbed5c2-cc8f-4300-bcc3-d139deafe54e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:39.443303 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-7cbed5c2-cc8f-4300-bcc3-d139deafe54e None None] MFA Rules not processed for user `755c2782cfb042f89950b8fc862e4fa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:25:39.444566 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-7cbed5c2-cc8f-4300-bcc3-d139deafe54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:39.547686 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 663/1351] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Jul 7 15:25:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:25:43.588656 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759387ab-858c-4f70-935d-9c68b53a830b None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:43.589162 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759387ab-858c-4f70-935d-9c68b53a830b None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:43.589243 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759387ab-858c-4f70-935d-9c68b53a830b None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:43.590809 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 689/1352] 10.222.0.25 () {54 vars in 921 bytes} [Thu Jul 7 15:25:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:25:43.596388 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64c36f-0b28-405c-95bc-ac5f5d125a6a None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:43.596860 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64c36f-0b28-405c-95bc-ac5f5d125a6a None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:43.597154 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64c36f-0b28-405c-95bc-ac5f5d125a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:43.616281 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-1c64c36f-0b28-405c-95bc-ac5f5d125a6a None None] MFA Rules not processed for user `755c2782cfb042f89950b8fc862e4fa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:25:43.617652 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-1c64c36f-0b28-405c-95bc-ac5f5d125a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:43.626768 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 664/1353] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Jul 7 15:25:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:25:44.340918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=13 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:25:45.277814 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11280a-2e04-4c20-9b0c-0ae9bde6221f None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:45.278438 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11280a-2e04-4c20-9b0c-0ae9bde6221f None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:45.278438 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11280a-2e04-4c20-9b0c-0ae9bde6221f None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:45.279808 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 690/1354] 10.222.0.25 () {54 vars in 921 bytes} [Thu Jul 7 15:25:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:25:45.285324 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27bd1a-3015-4cce-923e-5c884c768703 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:45.285732 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27bd1a-3015-4cce-923e-5c884c768703 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:45.285998 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27bd1a-3015-4cce-923e-5c884c768703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:45.304777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-fa27bd1a-3015-4cce-923e-5c884c768703 None None] MFA Rules not processed for user `755c2782cfb042f89950b8fc862e4fa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:25:45.306066 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-fa27bd1a-3015-4cce-923e-5c884c768703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:45.315048 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 665/1355] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Jul 7 15:25:45 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:25:48.877405 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0dbf3-5a49-4614-a5a3-4f4a4892aeb1 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:48.877941 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0dbf3-5a49-4614-a5a3-4f4a4892aeb1 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:48.877941 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0dbf3-5a49-4614-a5a3-4f4a4892aeb1 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:48.879439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 691/1356] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:25:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:25:48.886184 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6e200-dbb9-4354-a1de-62a928333941 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:48.887065 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6e200-dbb9-4354-a1de-62a928333941 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:48.887552 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6e200-dbb9-4354-a1de-62a928333941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:48.912174 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-dda6e200-dbb9-4354-a1de-62a928333941 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:25:48.913373 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-dda6e200-dbb9-4354-a1de-62a928333941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:48.922653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 666/1357] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:25:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:25:48.980956 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e222c5-1249-4cea-aaf9-f6a3893ea6c8 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:48.981488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e222c5-1249-4cea-aaf9-f6a3893ea6c8 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:48.981867 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e222c5-1249-4cea-aaf9-f6a3893ea6c8 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:48.984427 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 692/1358] 10.222.0.25 () {54 vars in 961 bytes} [Thu Jul 7 15:25:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:25:48.994041 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6018d-df63-4ca6-b1fd-ac72494a068f None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:49.001672 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6018d-df63-4ca6-b1fd-ac72494a068f None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:49.001672 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6018d-df63-4ca6-b1fd-ac72494a068f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:49.044387 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:25:49.053818 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-21c6018d-df63-4ca6-b1fd-ac72494a068f None None] MFA Rules not processed for user `de1632555cb345f2b9213da1925c02f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:25:49.055028 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-21c6018d-df63-4ca6-b1fd-ac72494a068f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:49.139197 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 667/1359] 10.222.0.25 () {58 vars in 1058 bytes} [Thu Jul 7 15:25:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:25:49.147866 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e222c5-1249-4cea-aaf9-f6a3893ea6c8 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:25:49.167516 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e222c5-1249-4cea-aaf9-f6a3893ea6c8 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:25:49.174578 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d709d87-4991-46be-b926-17a7ef7aaede service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:25:49.175307 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d709d87-4991-46be-b926-17a7ef7aaede service nova] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:49.175720 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d709d87-4991-46be-b926-17a7ef7aaede service nova] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:49.176072 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d709d87-4991-46be-b926-17a7ef7aaede service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:49.185140 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d709d87-4991-46be-b926-17a7ef7aaede service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:25:49.186843 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d709d87-4991-46be-b926-17a7ef7aaede service nova] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:25:49.197712 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 693/1360] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:25:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:25:51.433078 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c057aec-74e9-414a-b5cc-108eda3a178a None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:51.433589 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c057aec-74e9-414a-b5cc-108eda3a178a None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:51.433849 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c057aec-74e9-414a-b5cc-108eda3a178a None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:51.435485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 668/1361] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:25:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:25:51.441049 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff66485-7503-457d-9a6d-2cdd0371497f None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:51.441459 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff66485-7503-457d-9a6d-2cdd0371497f None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:51.441604 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff66485-7503-457d-9a6d-2cdd0371497f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:51.460045 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-8ff66485-7503-457d-9a6d-2cdd0371497f None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:25:51.461480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-8ff66485-7503-457d-9a6d-2cdd0371497f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:51.469544 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 694/1362] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:25:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:25:51.513009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c057aec-74e9-414a-b5cc-108eda3a178a None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:25:51.528503 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c057aec-74e9-414a-b5cc-108eda3a178a None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:25:51.534145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44531147-dbec-4551-aafa-7d24c34bcb16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:25:51.534914 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44531147-dbec-4551-aafa-7d24c34bcb16 service nova] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:51.535299 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44531147-dbec-4551-aafa-7d24c34bcb16 service nova] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:51.535812 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44531147-dbec-4551-aafa-7d24c34bcb16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:51.544462 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44531147-dbec-4551-aafa-7d24c34bcb16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:25:51.546781 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44531147-dbec-4551-aafa-7d24c34bcb16 service nova] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:25:51.556836 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 669/1363] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:25:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:25:54.044757 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b173ae3-f4da-4984-97f1-39940c2b28ec None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:54.045266 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b173ae3-f4da-4984-97f1-39940c2b28ec None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:54.045266 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b173ae3-f4da-4984-97f1-39940c2b28ec None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:54.046594 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 695/1364] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:25:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:25:54.051997 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac0222-7d72-4e6a-8f5b-00c8087808d0 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:54.052397 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac0222-7d72-4e6a-8f5b-00c8087808d0 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:54.052712 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac0222-7d72-4e6a-8f5b-00c8087808d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:54.070662 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-d4ac0222-7d72-4e6a-8f5b-00c8087808d0 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:25:54.071955 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-d4ac0222-7d72-4e6a-8f5b-00c8087808d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:54.080703 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 670/1365] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:25:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:25:54.127514 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a03a8-45ff-4ebd-b37d-83f98f491834 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:54.127832 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a03a8-45ff-4ebd-b37d-83f98f491834 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:54.128106 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a03a8-45ff-4ebd-b37d-83f98f491834 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:54.129469 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 696/1366] 10.222.0.25 () {54 vars in 961 bytes} [Thu Jul 7 15:25:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:25:54.135091 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507dca7c-8c37-4075-a30b-a67f52aaa0c7 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:54.135380 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507dca7c-8c37-4075-a30b-a67f52aaa0c7 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:54.141351 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507dca7c-8c37-4075-a30b-a67f52aaa0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:54.153995 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-507dca7c-8c37-4075-a30b-a67f52aaa0c7 None None] MFA Rules not processed for user `de1632555cb345f2b9213da1925c02f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:25:54.155179 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-507dca7c-8c37-4075-a30b-a67f52aaa0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:54.165525 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 671/1367] 10.222.0.25 () {58 vars in 1058 bytes} [Thu Jul 7 15:25:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:25:54.173293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718a03a8-45ff-4ebd-b37d-83f98f491834 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:25:54.188042 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718a03a8-45ff-4ebd-b37d-83f98f491834 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:25:54.193474 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc7fc9-0713-47b8-a94d-f514d2f42e82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:25:54.194168 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc7fc9-0713-47b8-a94d-f514d2f42e82 service nova] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:54.194512 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc7fc9-0713-47b8-a94d-f514d2f42e82 service nova] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:54.194870 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc7fc9-0713-47b8-a94d-f514d2f42e82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:54.203468 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bc7fc9-0713-47b8-a94d-f514d2f42e82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:25:54.205064 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bc7fc9-0713-47b8-a94d-f514d2f42e82 service nova] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:25:54.214368 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 697/1368] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:25:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:25:56.443298 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e624518-6ab8-4e7b-8801-9da55375e02c None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:56.443920 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e624518-6ab8-4e7b-8801-9da55375e02c None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:56.443920 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e624518-6ab8-4e7b-8801-9da55375e02c None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:56.445265 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 672/1369] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:25:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:25:56.450491 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e927114-ed01-4357-adf3-ebb714f7953f None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:56.450880 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e927114-ed01-4357-adf3-ebb714f7953f None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:56.451034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e927114-ed01-4357-adf3-ebb714f7953f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:56.471313 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-0e927114-ed01-4357-adf3-ebb714f7953f None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:25:56.472695 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-0e927114-ed01-4357-adf3-ebb714f7953f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:56.480970 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 698/1370] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:25:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:25:56.525257 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e624518-6ab8-4e7b-8801-9da55375e02c None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:25:56.540850 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e624518-6ab8-4e7b-8801-9da55375e02c None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:25:56.547070 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d9702a-b575-4dab-9106-d541888ba605 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:25:56.547818 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d9702a-b575-4dab-9106-d541888ba605 service nova] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:56.548170 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d9702a-b575-4dab-9106-d541888ba605 service nova] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:56.548512 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d9702a-b575-4dab-9106-d541888ba605 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:56.557298 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d9702a-b575-4dab-9106-d541888ba605 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:25:56.558980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d9702a-b575-4dab-9106-d541888ba605 service nova] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:25:56.569434 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 673/1371] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:25:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:25:56.994299 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8618e-4aec-4c99-b1de-d1bd39e218a7 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:56.994836 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8618e-4aec-4c99-b1de-d1bd39e218a7 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:56.995266 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8618e-4aec-4c99-b1de-d1bd39e218a7 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:56.996919 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 699/1372] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:25:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:25:57.002397 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d504d-2476-4112-b156-99516f9bd6df None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:57.002802 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d504d-2476-4112-b156-99516f9bd6df None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:57.003191 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d504d-2476-4112-b156-99516f9bd6df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:57.022049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-750d504d-2476-4112-b156-99516f9bd6df None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:25:57.023391 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-750d504d-2476-4112-b156-99516f9bd6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:57.032648 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 674/1373] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:25:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:25:57.376774 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e54cf5-645e-4103-904c-89527981f725 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:57.376774 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e54cf5-645e-4103-904c-89527981f725 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:57.376774 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e54cf5-645e-4103-904c-89527981f725 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:57.379706 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 700/1374] 10.222.0.25 () {54 vars in 963 bytes} [Thu Jul 7 15:25:57 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:25:57.386986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5bcfa2-f839-4810-8c90-8affe0335747 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:57.387384 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5bcfa2-f839-4810-8c90-8affe0335747 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:57.387791 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5bcfa2-f839-4810-8c90-8affe0335747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:57.446572 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-0b5bcfa2-f839-4810-8c90-8affe0335747 None None] MFA Rules not processed for user `2a0fbebb3dd2458ba4fe5d7f0ce211b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:25:57.447952 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-0b5bcfa2-f839-4810-8c90-8affe0335747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:57.520610 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 675/1375] 10.222.0.25 () {58 vars in 1060 bytes} [Thu Jul 7 15:25:57 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:25:57.528079 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e54cf5-645e-4103-904c-89527981f725 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:25:57.539646 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e54cf5-645e-4103-904c-89527981f725 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:25:57.543285 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c1f793-7f95-41f1-86aa-9408a7834364 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:25:57.544058 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1f793-7f95-41f1-86aa-9408a7834364 service cinder] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:57.544457 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1f793-7f95-41f1-86aa-9408a7834364 service cinder] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:57.544825 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1f793-7f95-41f1-86aa-9408a7834364 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:57.555354 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c1f793-7f95-41f1-86aa-9408a7834364 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:25:57.557067 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c1f793-7f95-41f1-86aa-9408a7834364 service cinder] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:25:57.567591 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 701/1376] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:25:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:25:58.775437 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493be688-49ee-4e98-8a33-25831b60404b None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:58.776009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493be688-49ee-4e98-8a33-25831b60404b None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:58.776101 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493be688-49ee-4e98-8a33-25831b60404b None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:58.777471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 676/1377] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:25:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:25:58.783062 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2115a5-4412-408f-96b8-3fe0e468ea2d None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:58.783505 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2115a5-4412-408f-96b8-3fe0e468ea2d None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:58.783643 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2115a5-4412-408f-96b8-3fe0e468ea2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:58.801444 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-0a2115a5-4412-408f-96b8-3fe0e468ea2d None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:25:58.802638 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-0a2115a5-4412-408f-96b8-3fe0e468ea2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:25:58.810561 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 702/1378] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:25:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:25:58.853451 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493be688-49ee-4e98-8a33-25831b60404b None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:25:58.868372 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493be688-49ee-4e98-8a33-25831b60404b None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:25:58.873756 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d757cc8-9646-49a3-89f3-34631b935160 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:25:58.874391 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d757cc8-9646-49a3-89f3-34631b935160 service nova] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:25:58.874674 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d757cc8-9646-49a3-89f3-34631b935160 service nova] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:25:58.874951 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d757cc8-9646-49a3-89f3-34631b935160 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:25:58.883023 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d757cc8-9646-49a3-89f3-34631b935160 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:25:58.884643 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d757cc8-9646-49a3-89f3-34631b935160 service nova] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:25:58.888849 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:25:58.895077 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 677/1379] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:25:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:26:01.210276 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b2999-eea3-408f-bbcf-489c82a1f5cb None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:01.211121 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b2999-eea3-408f-bbcf-489c82a1f5cb None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:01.211399 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b2999-eea3-408f-bbcf-489c82a1f5cb None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:01.212954 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 703/1380] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:26:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:26:01.218605 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8495d77-1196-47db-aa0f-3c4fd3899bae None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:01.219479 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8495d77-1196-47db-aa0f-3c4fd3899bae None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:01.220012 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8495d77-1196-47db-aa0f-3c4fd3899bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:01.243665 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-f8495d77-1196-47db-aa0f-3c4fd3899bae None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:26:01.245152 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-f8495d77-1196-47db-aa0f-3c4fd3899bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:26:01.258380 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 678/1381] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:26:01.308278 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b2999-eea3-408f-bbcf-489c82a1f5cb None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:26:01.323439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b2999-eea3-408f-bbcf-489c82a1f5cb None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:26:01.329853 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483c8dd5-4766-42d9-bde4-47a8192972b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:26:01.330497 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c8dd5-4766-42d9-bde4-47a8192972b9 service nova] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:01.330783 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c8dd5-4766-42d9-bde4-47a8192972b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:01.331057 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c8dd5-4766-42d9-bde4-47a8192972b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:01.339541 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483c8dd5-4766-42d9-bde4-47a8192972b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:26:01.341152 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483c8dd5-4766-42d9-bde4-47a8192972b9 service nova] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:26:01.351439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 704/1382] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:26:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:26:03.539772 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16646137-c53c-4b98-8bbe-7139ab2b26c9 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:03.540274 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16646137-c53c-4b98-8bbe-7139ab2b26c9 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:03.540373 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16646137-c53c-4b98-8bbe-7139ab2b26c9 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:03.541665 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 679/1383] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:26:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:26:03.547550 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4cede3-1995-47e2-aafa-7f6a11c2ce91 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:03.548014 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4cede3-1995-47e2-aafa-7f6a11c2ce91 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:03.548253 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4cede3-1995-47e2-aafa-7f6a11c2ce91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:03.567036 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-cf4cede3-1995-47e2-aafa-7f6a11c2ce91 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:26:03.568338 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-cf4cede3-1995-47e2-aafa-7f6a11c2ce91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:26:03.577176 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 705/1384] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:26:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:26:03.618735 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16646137-c53c-4b98-8bbe-7139ab2b26c9 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:26:03.634678 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16646137-c53c-4b98-8bbe-7139ab2b26c9 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:26:03.640676 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09e527-9fc1-4b81-927c-814949bfdf00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:26:03.641323 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09e527-9fc1-4b81-927c-814949bfdf00 service nova] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:03.641608 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09e527-9fc1-4b81-927c-814949bfdf00 service nova] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:03.641887 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09e527-9fc1-4b81-927c-814949bfdf00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:03.650502 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb09e527-9fc1-4b81-927c-814949bfdf00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:26:03.652111 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb09e527-9fc1-4b81-927c-814949bfdf00 service nova] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:26:03.661971 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 680/1385] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:26:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:26:05.779180 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933613c-b2db-461c-87ce-da77ff59c669 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:05.779784 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933613c-b2db-461c-87ce-da77ff59c669 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:05.779943 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933613c-b2db-461c-87ce-da77ff59c669 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:05.781776 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 706/1386] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:26:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:26:05.788433 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552914f-9dab-49bd-b068-57dc174f21e8 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:05.789127 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552914f-9dab-49bd-b068-57dc174f21e8 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:05.789127 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552914f-9dab-49bd-b068-57dc174f21e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:05.816600 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-2552914f-9dab-49bd-b068-57dc174f21e8 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:26:05.818589 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-2552914f-9dab-49bd-b068-57dc174f21e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:26:05.828393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 681/1387] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:26:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:26:05.872186 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2933613c-b2db-461c-87ce-da77ff59c669 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:26:05.887254 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2933613c-b2db-461c-87ce-da77ff59c669 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:26:05.892774 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061a0cc-3d7a-4393-9b1b-a21bbf55df1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:26:05.893414 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061a0cc-3d7a-4393-9b1b-a21bbf55df1e service nova] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:05.893702 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061a0cc-3d7a-4393-9b1b-a21bbf55df1e service nova] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:05.893971 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061a0cc-3d7a-4393-9b1b-a21bbf55df1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:05.902307 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b061a0cc-3d7a-4393-9b1b-a21bbf55df1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:26:05.903927 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b061a0cc-3d7a-4393-9b1b-a21bbf55df1e service nova] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:26:05.913665 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 707/1388] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:26:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:26:08.484821 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42bacfd-89e1-4f40-a29b-221aee5eea0b None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:08.485366 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42bacfd-89e1-4f40-a29b-221aee5eea0b None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:08.485366 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42bacfd-89e1-4f40-a29b-221aee5eea0b None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:08.486863 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 682/1389] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:26:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:26:08.492434 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12265fda-bc6d-4daf-b78d-c8d83c24f661 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:08.492878 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12265fda-bc6d-4daf-b78d-c8d83c24f661 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:08.493070 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12265fda-bc6d-4daf-b78d-c8d83c24f661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:08.511043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-12265fda-bc6d-4daf-b78d-c8d83c24f661 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:26:08.512228 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-12265fda-bc6d-4daf-b78d-c8d83c24f661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:26:08.521604 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 708/1390] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:26:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:26:08.565779 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42bacfd-89e1-4f40-a29b-221aee5eea0b None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:26:08.586807 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42bacfd-89e1-4f40-a29b-221aee5eea0b None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:26:08.594593 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f952c9f-50b1-40f9-bff8-5a0ef1c75566 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:26:08.595457 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f952c9f-50b1-40f9-bff8-5a0ef1c75566 service nova] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:08.595879 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f952c9f-50b1-40f9-bff8-5a0ef1c75566 service nova] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:08.596255 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f952c9f-50b1-40f9-bff8-5a0ef1c75566 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:08.607472 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f952c9f-50b1-40f9-bff8-5a0ef1c75566 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:26:08.610314 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f952c9f-50b1-40f9-bff8-5a0ef1c75566 service nova] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:26:08.622729 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 683/1391] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:26:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:26:10.901694 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fe7dc-b336-4d24-bd40-fd6203d72160 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:10.902480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fe7dc-b336-4d24-bd40-fd6203d72160 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:10.902480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fe7dc-b336-4d24-bd40-fd6203d72160 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:10.903797 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 709/1392] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:26:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:26:10.909871 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba88ff-d67d-497b-9b3a-d3bbf4fac517 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:10.910624 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba88ff-d67d-497b-9b3a-d3bbf4fac517 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:10.911121 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba88ff-d67d-497b-9b3a-d3bbf4fac517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:10.933765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-cbba88ff-d67d-497b-9b3a-d3bbf4fac517 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:26:10.934973 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-cbba88ff-d67d-497b-9b3a-d3bbf4fac517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:26:10.943408 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 684/1393] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:26:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:26:10.989537 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761fe7dc-b336-4d24-bd40-fd6203d72160 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:26:11.006855 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761fe7dc-b336-4d24-bd40-fd6203d72160 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:26:11.010970 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ddae73-3742-4141-ac33-bbbc4c3ec7f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:26:11.011814 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ddae73-3742-4141-ac33-bbbc4c3ec7f4 service nova] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:11.012099 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ddae73-3742-4141-ac33-bbbc4c3ec7f4 service nova] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:11.012419 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ddae73-3742-4141-ac33-bbbc4c3ec7f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:11.021990 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ddae73-3742-4141-ac33-bbbc4c3ec7f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:26:11.023891 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ddae73-3742-4141-ac33-bbbc4c3ec7f4 service nova] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:26:11.033159 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 710/1394] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:26:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:26:12.954225 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f23705-e3c6-4878-bf05-7c3e30c0f5a8 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:12.954778 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f23705-e3c6-4878-bf05-7c3e30c0f5a8 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:12.954778 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f23705-e3c6-4878-bf05-7c3e30c0f5a8 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:12.956451 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 685/1395] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:26:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:26:12.961668 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c034c2-ea9b-4f0a-8f66-60b02747332f None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:12.962113 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c034c2-ea9b-4f0a-8f66-60b02747332f None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:12.962243 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c034c2-ea9b-4f0a-8f66-60b02747332f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:12.980313 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-d0c034c2-ea9b-4f0a-8f66-60b02747332f None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:26:12.981500 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-d0c034c2-ea9b-4f0a-8f66-60b02747332f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:26:12.991687 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 711/1396] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:26:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:26:13.053901 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f3c4a5-b8f8-4b89-9232-4273281ff5b0 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:13.054293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f3c4a5-b8f8-4b89-9232-4273281ff5b0 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:13.054727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f3c4a5-b8f8-4b89-9232-4273281ff5b0 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:13.057017 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 686/1397] 10.222.0.25 () {54 vars in 962 bytes} [Thu Jul 7 15:26:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:26:13.063039 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8feae8-29f4-4e18-b790-2628554ceb7b None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:13.063448 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8feae8-29f4-4e18-b790-2628554ceb7b None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:13.064062 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8feae8-29f4-4e18-b790-2628554ceb7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:13.126879 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-4c8feae8-29f4-4e18-b790-2628554ceb7b None None] MFA Rules not processed for user `3d0d29e718f446e2a9fd1b2575a86041`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:26:13.128250 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-4c8feae8-29f4-4e18-b790-2628554ceb7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:26:13.181368 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ab8d1-1746-43e4-a8ad-7c14ed303cd0 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:13.181654 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ab8d1-1746-43e4-a8ad-7c14ed303cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:13.181930 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ab8d1-1746-43e4-a8ad-7c14ed303cd0 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:13.183307 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 687/1398] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:26:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:26:13.188795 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ebc31e-c29e-43fd-9f21-aaa2274fbb25 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:13.189083 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ebc31e-c29e-43fd-9f21-aaa2274fbb25 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:13.189359 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ebc31e-c29e-43fd-9f21-aaa2274fbb25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:13.191047 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 712/1399] 10.222.0.25 () {58 vars in 1059 bytes} [Thu Jul 7 15:26:13 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:26:13.198535 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8feae8-29f4-4e18-b790-2628554ceb7b None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:26:13.228375 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8feae8-29f4-4e18-b790-2628554ceb7b None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:26:13.228526 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-e9ebc31e-c29e-43fd-9f21-aaa2274fbb25 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:26:13.228639 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d601250-1363-4d9c-8195-5454cb8da6c6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0d29e718f446e2a9fd1b2575a86041', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:26:13.228639 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d601250-1363-4d9c-8195-5454cb8da6c6 service glance] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:13.228842 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-e9ebc31e-c29e-43fd-9f21-aaa2274fbb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:26:13.228941 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d601250-1363-4d9c-8195-5454cb8da6c6 service glance] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:13.228941 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d601250-1363-4d9c-8195-5454cb8da6c6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:13.229114 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 688/1400] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:26:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:26:13.229213 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d601250-1363-4d9c-8195-5454cb8da6c6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:26:13.229213 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d601250-1363-4d9c-8195-5454cb8da6c6 service glance] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:26:13.231779 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 713/1401] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:26:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 15:26:13.271528 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ebc31e-c29e-43fd-9f21-aaa2274fbb25 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:26:13.284496 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ebc31e-c29e-43fd-9f21-aaa2274fbb25 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:26:13.291311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b24b1-d450-4232-9dd7-43c4f47359bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:26:13.291973 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b24b1-d450-4232-9dd7-43c4f47359bd service nova] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:13.292255 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b24b1-d450-4232-9dd7-43c4f47359bd service nova] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:13.292527 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b24b1-d450-4232-9dd7-43c4f47359bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:13.301651 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3b24b1-d450-4232-9dd7-43c4f47359bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:26:13.303208 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3b24b1-d450-4232-9dd7-43c4f47359bd service nova] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:26:13.313337 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 689/1402] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:26:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:26:14.098469 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d23ac6-ba3a-436a-8fd9-3c81bfd4b224 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:14.098931 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d23ac6-ba3a-436a-8fd9-3c81bfd4b224 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:14.099029 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d23ac6-ba3a-436a-8fd9-3c81bfd4b224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:14.105192 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:26:14.149642 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-49d23ac6-ba3a-436a-8fd9-3c81bfd4b224 None None] MFA Rules not processed for user `cd3c04f39d774998b66e6ce920925691`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:26:14.150854 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-49d23ac6-ba3a-436a-8fd9-3c81bfd4b224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:26:14.214814 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 714/1403] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Jul 7 15:26:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:26:14.257346 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829951d-4adf-4c35-b6e6-ae0b1e7510c6 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:14.263062 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829951d-4adf-4c35-b6e6-ae0b1e7510c6 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:14.263062 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829951d-4adf-4c35-b6e6-ae0b1e7510c6 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:14.263062 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 690/1404] 10.222.0.25 () {54 vars in 942 bytes} [Thu Jul 7 15:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:26:14.265624 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3da887-c297-4d03-83dd-461afd523c9c None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:14.265941 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3da887-c297-4d03-83dd-461afd523c9c None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:14.266248 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3da887-c297-4d03-83dd-461afd523c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:14.315294 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-0d3da887-c297-4d03-83dd-461afd523c9c None None] MFA Rules not processed for user `b18033f849b240999d12c1885d2f7b91`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:26:14.316591 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-0d3da887-c297-4d03-83dd-461afd523c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:26:14.388517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 715/1405] 10.222.0.25 () {58 vars in 1039 bytes} [Thu Jul 7 15:26:14 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:26:14.397662 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829951d-4adf-4c35-b6e6-ae0b1e7510c6 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:26:14.410993 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829951d-4adf-4c35-b6e6-ae0b1e7510c6 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:26:14.415888 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e8d9c-2c7e-440b-b4fe-4ca821639033 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b18033f849b240999d12c1885d2f7b91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:26:14.416515 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e8d9c-2c7e-440b-b4fe-4ca821639033 service swift] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:14.416794 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e8d9c-2c7e-440b-b4fe-4ca821639033 service swift] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:14.417066 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e8d9c-2c7e-440b-b4fe-4ca821639033 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:14.425775 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e8d9c-2c7e-440b-b4fe-4ca821639033 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:26:14.427413 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e8d9c-2c7e-440b-b4fe-4ca821639033 service swift] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:26:14.435373 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 691/1406] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Jul 7 15:26:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 07 15:26:15.508445 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463328b-f91e-4e9f-9db8-8151f111b113 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:15.508998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463328b-f91e-4e9f-9db8-8151f111b113 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:15.509804 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463328b-f91e-4e9f-9db8-8151f111b113 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:15.518046 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 716/1407] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:26:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:26:15.520937 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe35ec-9c73-4bae-8709-ca9c1f369c49 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:15.522299 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe35ec-9c73-4bae-8709-ca9c1f369c49 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:15.523065 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe35ec-9c73-4bae-8709-ca9c1f369c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:15.546250 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-9dfe35ec-9c73-4bae-8709-ca9c1f369c49 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:26:15.550161 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-9dfe35ec-9c73-4bae-8709-ca9c1f369c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:26:15.558405 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 692/1408] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:26:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:26:15.602355 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1463328b-f91e-4e9f-9db8-8151f111b113 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:26:15.617291 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1463328b-f91e-4e9f-9db8-8151f111b113 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:26:15.627143 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948cb343-3961-4636-a110-13c21c43fcd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:26:15.627820 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948cb343-3961-4636-a110-13c21c43fcd5 service nova] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:15.628105 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948cb343-3961-4636-a110-13c21c43fcd5 service nova] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:15.628380 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948cb343-3961-4636-a110-13c21c43fcd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:15.637540 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948cb343-3961-4636-a110-13c21c43fcd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:26:15.639745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948cb343-3961-4636-a110-13c21c43fcd5 service nova] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:26:15.649282 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 717/1409] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:26:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:26:25.540808 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=40 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:26:25.645479 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=88 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:26:38.163297 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba6e0d-6a15-452a-a2a2-75170556150e None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:38.163860 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba6e0d-6a15-452a-a2a2-75170556150e None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:38.163860 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba6e0d-6a15-452a-a2a2-75170556150e None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:38.165223 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 693/1410] 10.222.0.25 () {54 vars in 918 bytes} [Thu Jul 7 15:26:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:26:38.170391 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54958e34-12cb-4b78-9642-8ab34d641f15 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:38.170801 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54958e34-12cb-4b78-9642-8ab34d641f15 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:38.170940 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54958e34-12cb-4b78-9642-8ab34d641f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:38.188447 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-54958e34-12cb-4b78-9642-8ab34d641f15 None None] MFA Rules not processed for user `755c2782cfb042f89950b8fc862e4fa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:26:38.189647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-54958e34-12cb-4b78-9642-8ab34d641f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:26:38.198018 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 718/1411] 10.222.0.25 () {58 vars in 1015 bytes} [Thu Jul 7 15:26:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:26:38.212405 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79cdec-1c34-4259-910e-ebe0969b8c0a None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:38.212689 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79cdec-1c34-4259-910e-ebe0969b8c0a None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:38.212964 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79cdec-1c34-4259-910e-ebe0969b8c0a None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:38.214293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 694/1412] 10.222.0.25 () {54 vars in 918 bytes} [Thu Jul 7 15:26:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:26:38.219640 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e150404-2ce1-444a-a723-91d6ceec1979 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:38.219877 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e150404-2ce1-444a-a723-91d6ceec1979 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:38.220155 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e150404-2ce1-444a-a723-91d6ceec1979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:38.238319 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-0e150404-2ce1-444a-a723-91d6ceec1979 None None] MFA Rules not processed for user `de1632555cb345f2b9213da1925c02f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:26:38.239532 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-0e150404-2ce1-444a-a723-91d6ceec1979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:26:38.248539 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 719/1413] 10.222.0.25 () {58 vars in 1015 bytes} [Thu Jul 7 15:26:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:26:40.720036 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad44df-0d81-4886-8ca7-1eaf88e99a7d None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:40.720594 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad44df-0d81-4886-8ca7-1eaf88e99a7d None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:40.720594 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad44df-0d81-4886-8ca7-1eaf88e99a7d None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:40.721976 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 695/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 7 15:26:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:26:40.727374 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6d644-2282-480e-9558-139ce6db9728 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:40.727811 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6d644-2282-480e-9558-139ce6db9728 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:40.727957 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6d644-2282-480e-9558-139ce6db9728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:40.745805 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-e1d6d644-2282-480e-9558-139ce6db9728 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:26:40.747527 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-e1d6d644-2282-480e-9558-139ce6db9728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:26:40.755172 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 720/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 7 15:26:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:26:40.798970 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ad44df-0d81-4886-8ca7-1eaf88e99a7d None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:26:40.817598 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ad44df-0d81-4886-8ca7-1eaf88e99a7d None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:26:40.823106 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f1521-64bc-4f10-b0be-8132dd1041b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:26:40.823769 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f1521-64bc-4f10-b0be-8132dd1041b8 service nova] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:40.824075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f1521-64bc-4f10-b0be-8132dd1041b8 service nova] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:40.824349 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f1521-64bc-4f10-b0be-8132dd1041b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:40.833319 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f1521-64bc-4f10-b0be-8132dd1041b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:26:40.834898 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f1521-64bc-4f10-b0be-8132dd1041b8 service nova] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:26:40.844679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 696/1416] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:26:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:26:41.736347 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ca675-ed1d-4882-b8cf-1a985f534422 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:41.736881 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ca675-ed1d-4882-b8cf-1a985f534422 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:41.736881 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ca675-ed1d-4882-b8cf-1a985f534422 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:41.738164 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 721/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 7 15:26:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:26:41.743262 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5937ee-36ed-413b-9c82-505ad4c9f5ac None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:41.743717 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5937ee-36ed-413b-9c82-505ad4c9f5ac None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:41.743857 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5937ee-36ed-413b-9c82-505ad4c9f5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:41.761880 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-db5937ee-36ed-413b-9c82-505ad4c9f5ac None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:26:41.763716 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-db5937ee-36ed-413b-9c82-505ad4c9f5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:26:41.771431 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 697/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 7 15:26:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:26:41.919657 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ca675-ed1d-4882-b8cf-1a985f534422 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:26:41.933460 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ca675-ed1d-4882-b8cf-1a985f534422 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:26:41.939146 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ddf37-5c66-427d-8023-84bc45c578aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:26:41.939814 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ddf37-5c66-427d-8023-84bc45c578aa service nova] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:41.940096 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ddf37-5c66-427d-8023-84bc45c578aa service nova] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:41.940369 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ddf37-5c66-427d-8023-84bc45c578aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:41.948519 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7ddf37-5c66-427d-8023-84bc45c578aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:26:41.950637 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7ddf37-5c66-427d-8023-84bc45c578aa service nova] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:26:41.960172 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 722/1419] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:26:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:26:42.833097 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149178ac-5949-4630-b44c-95c47bed820d None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:42.833641 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149178ac-5949-4630-b44c-95c47bed820d None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:42.833641 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149178ac-5949-4630-b44c-95c47bed820d None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:42.835030 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 698/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 7 15:26:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:26:42.839883 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc1dfd-303d-4f08-b83c-c0e0eb7048fe None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:42.840288 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc1dfd-303d-4f08-b83c-c0e0eb7048fe None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:42.840465 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc1dfd-303d-4f08-b83c-c0e0eb7048fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:42.859008 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-98cc1dfd-303d-4f08-b83c-c0e0eb7048fe None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:26:42.860215 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-98cc1dfd-303d-4f08-b83c-c0e0eb7048fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:26:42.867851 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 723/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 7 15:26:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:26:43.017643 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149178ac-5949-4630-b44c-95c47bed820d None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:26:43.032016 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149178ac-5949-4630-b44c-95c47bed820d None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:26:43.037337 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d668b-11bd-47a6-a642-e5bb5d94547c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:26:43.037999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d668b-11bd-47a6-a642-e5bb5d94547c service nova] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:43.038282 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d668b-11bd-47a6-a642-e5bb5d94547c service nova] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:43.038640 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d668b-11bd-47a6-a642-e5bb5d94547c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:43.047053 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d668b-11bd-47a6-a642-e5bb5d94547c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:26:43.048664 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d668b-11bd-47a6-a642-e5bb5d94547c service nova] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:26:43.058023 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 699/1422] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:26:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:26:43.924280 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96068c97-ef29-4095-aa4b-27b49f0257a6 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:43.924812 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96068c97-ef29-4095-aa4b-27b49f0257a6 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:43.924812 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96068c97-ef29-4095-aa4b-27b49f0257a6 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:43.926221 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 724/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 7 15:26:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:26:43.931101 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1664a-51b4-4fae-a583-a50c47fdc24b None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:43.931520 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1664a-51b4-4fae-a583-a50c47fdc24b None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:43.931698 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1664a-51b4-4fae-a583-a50c47fdc24b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:43.949251 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-a4c1664a-51b4-4fae-a583-a50c47fdc24b None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:26:43.950515 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-a4c1664a-51b4-4fae-a583-a50c47fdc24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:26:43.958383 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 700/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 7 15:26:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:26:44.111955 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96068c97-ef29-4095-aa4b-27b49f0257a6 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:26:44.126196 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96068c97-ef29-4095-aa4b-27b49f0257a6 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:26:44.131538 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be59288-2562-41ae-a6f0-0d7dd60c5552 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:26:44.132219 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be59288-2562-41ae-a6f0-0d7dd60c5552 service nova] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:44.132486 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be59288-2562-41ae-a6f0-0d7dd60c5552 service nova] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:44.132785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be59288-2562-41ae-a6f0-0d7dd60c5552 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:44.141919 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be59288-2562-41ae-a6f0-0d7dd60c5552 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:26:44.143505 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be59288-2562-41ae-a6f0-0d7dd60c5552 service nova] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:26:44.153642 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 725/1425] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:26:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:26:45.015037 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee640e-ddea-4792-a35a-e9165cb7ee19 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:45.015656 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee640e-ddea-4792-a35a-e9165cb7ee19 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:45.015656 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee640e-ddea-4792-a35a-e9165cb7ee19 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:45.016997 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 701/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 7 15:26:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:26:45.021850 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5913f2-9be6-48a5-a658-4d6fda58196c None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:45.022290 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5913f2-9be6-48a5-a658-4d6fda58196c None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:45.022398 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5913f2-9be6-48a5-a658-4d6fda58196c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:45.040409 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-8f5913f2-9be6-48a5-a658-4d6fda58196c None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:26:45.041616 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-8f5913f2-9be6-48a5-a658-4d6fda58196c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:26:45.049393 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 726/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 7 15:26:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:26:45.199680 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee640e-ddea-4792-a35a-e9165cb7ee19 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:26:45.213735 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee640e-ddea-4792-a35a-e9165cb7ee19 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:26:45.221608 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996f524f-9d5e-4afa-9bc8-4409be5576a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:26:45.221608 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f524f-9d5e-4afa-9bc8-4409be5576a7 service nova] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:45.221608 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f524f-9d5e-4afa-9bc8-4409be5576a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:45.221608 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f524f-9d5e-4afa-9bc8-4409be5576a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:45.229680 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996f524f-9d5e-4afa-9bc8-4409be5576a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:26:45.231931 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996f524f-9d5e-4afa-9bc8-4409be5576a7 service nova] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:26:45.241794 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 702/1428] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:26:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:26:46.118459 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb41b2b-89d0-474f-9a60-8665dc8dcd11 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:46.119021 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb41b2b-89d0-474f-9a60-8665dc8dcd11 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:46.119021 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb41b2b-89d0-474f-9a60-8665dc8dcd11 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:46.120404 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 727/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 7 15:26:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:26:46.125745 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059acfe-6229-4392-9476-cbec26de7c53 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:46.126168 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059acfe-6229-4392-9476-cbec26de7c53 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:46.126363 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059acfe-6229-4392-9476-cbec26de7c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:46.145031 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-2059acfe-6229-4392-9476-cbec26de7c53 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:26:46.147089 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-2059acfe-6229-4392-9476-cbec26de7c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:26:46.158425 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 703/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 7 15:26:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:26:46.309489 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb41b2b-89d0-474f-9a60-8665dc8dcd11 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:26:46.323336 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb41b2b-89d0-474f-9a60-8665dc8dcd11 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:26:46.329224 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9e4be1-7f9e-41ac-af10-c7b633d76a60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:26:46.329854 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e4be1-7f9e-41ac-af10-c7b633d76a60 service nova] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:46.330137 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e4be1-7f9e-41ac-af10-c7b633d76a60 service nova] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:46.330414 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e4be1-7f9e-41ac-af10-c7b633d76a60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:46.338351 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9e4be1-7f9e-41ac-af10-c7b633d76a60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:26:46.339964 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9e4be1-7f9e-41ac-af10-c7b633d76a60 service nova] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:26:46.349821 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 728/1431] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:26:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:26:47.242622 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79152e5-6db9-487f-bc39-5b70a1d5b459 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:47.243163 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79152e5-6db9-487f-bc39-5b70a1d5b459 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:47.243163 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79152e5-6db9-487f-bc39-5b70a1d5b459 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:47.244480 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 704/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 7 15:26:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:26:47.249420 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d255195-d3ec-4c58-a11c-9db2249874e0 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:47.249856 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d255195-d3ec-4c58-a11c-9db2249874e0 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:47.249965 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d255195-d3ec-4c58-a11c-9db2249874e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:47.268097 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-5d255195-d3ec-4c58-a11c-9db2249874e0 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:26:47.269295 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-5d255195-d3ec-4c58-a11c-9db2249874e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:26:47.277027 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 729/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 7 15:26:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:26:47.426045 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79152e5-6db9-487f-bc39-5b70a1d5b459 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:26:47.440695 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79152e5-6db9-487f-bc39-5b70a1d5b459 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:26:47.446024 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c11b1f4-7eff-4acc-921b-f3cdeeb56f08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:26:47.446651 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c11b1f4-7eff-4acc-921b-f3cdeeb56f08 service nova] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:47.446932 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c11b1f4-7eff-4acc-921b-f3cdeeb56f08 service nova] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:47.447210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c11b1f4-7eff-4acc-921b-f3cdeeb56f08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:47.455284 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c11b1f4-7eff-4acc-921b-f3cdeeb56f08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:26:47.457012 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c11b1f4-7eff-4acc-921b-f3cdeeb56f08 service nova] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:26:47.467118 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 705/1434] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:26:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:26:48.376733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa995f61-597b-4838-b35b-7da1e1de679a None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:48.377290 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa995f61-597b-4838-b35b-7da1e1de679a None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:48.377408 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa995f61-597b-4838-b35b-7da1e1de679a None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:48.378665 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 730/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 7 15:26:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:26:48.383736 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84aac47-9e61-427a-90fd-756cd2a6cb49 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:48.384135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84aac47-9e61-427a-90fd-756cd2a6cb49 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:48.384292 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84aac47-9e61-427a-90fd-756cd2a6cb49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:48.403035 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-e84aac47-9e61-427a-90fd-756cd2a6cb49 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:26:48.404281 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-e84aac47-9e61-427a-90fd-756cd2a6cb49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:26:48.412965 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 706/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 7 15:26:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:26:48.563516 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa995f61-597b-4838-b35b-7da1e1de679a None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:26:48.577734 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa995f61-597b-4838-b35b-7da1e1de679a None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:26:48.583135 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b205a6-293c-441e-a63b-88a5e5699cd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:26:48.583805 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b205a6-293c-441e-a63b-88a5e5699cd4 service nova] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:48.584091 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b205a6-293c-441e-a63b-88a5e5699cd4 service nova] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:48.584369 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b205a6-293c-441e-a63b-88a5e5699cd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:48.592398 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b205a6-293c-441e-a63b-88a5e5699cd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:26:48.593970 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b205a6-293c-441e-a63b-88a5e5699cd4 service nova] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:26:48.603254 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 731/1437] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:26:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:26:49.452400 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90939a-8ddf-404f-9d44-77f8c5e2771e None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:49.452898 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90939a-8ddf-404f-9d44-77f8c5e2771e None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:49.453007 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90939a-8ddf-404f-9d44-77f8c5e2771e None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:49.454345 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 707/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 7 15:26:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:26:49.459505 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f425c96-35e4-4f6b-952d-96f21b6f6fec None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:49.459960 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f425c96-35e4-4f6b-952d-96f21b6f6fec None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:49.460098 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f425c96-35e4-4f6b-952d-96f21b6f6fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:49.477828 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-0f425c96-35e4-4f6b-952d-96f21b6f6fec None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:26:49.479030 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-0f425c96-35e4-4f6b-952d-96f21b6f6fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:26:49.487602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 732/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 7 15:26:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:26:49.635652 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90939a-8ddf-404f-9d44-77f8c5e2771e None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:26:49.649599 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90939a-8ddf-404f-9d44-77f8c5e2771e None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:26:49.654828 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e60296-3330-4e2b-af71-9f0922d288ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:26:49.655454 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e60296-3330-4e2b-af71-9f0922d288ec service nova] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:49.655762 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e60296-3330-4e2b-af71-9f0922d288ec service nova] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:49.656037 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e60296-3330-4e2b-af71-9f0922d288ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:49.663939 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e60296-3330-4e2b-af71-9f0922d288ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:26:49.665480 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e60296-3330-4e2b-af71-9f0922d288ec service nova] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:26:49.675078 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 708/1440] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:26:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:26:51.577742 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23bb941-a82d-4723-946f-dc6f25d8be39 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:51.578297 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23bb941-a82d-4723-946f-dc6f25d8be39 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:51.578297 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23bb941-a82d-4723-946f-dc6f25d8be39 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:51.579756 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 733/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 7 15:26:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:26:51.585000 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f43fc-69f1-49cf-8926-8f2e062f6fc1 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:51.585410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f43fc-69f1-49cf-8926-8f2e062f6fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:51.585549 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f43fc-69f1-49cf-8926-8f2e062f6fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:51.603328 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-a64f43fc-69f1-49cf-8926-8f2e062f6fc1 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:26:51.604542 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-a64f43fc-69f1-49cf-8926-8f2e062f6fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:26:51.612520 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 709/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 7 15:26:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:26:51.649722 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23bb941-a82d-4723-946f-dc6f25d8be39 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:26:51.661030 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23bb941-a82d-4723-946f-dc6f25d8be39 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:26:51.664655 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664889ee-8658-49b2-b0e0-a027ededa8ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:26:51.665325 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664889ee-8658-49b2-b0e0-a027ededa8ca service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:51.665607 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664889ee-8658-49b2-b0e0-a027ededa8ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:51.665884 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664889ee-8658-49b2-b0e0-a027ededa8ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:51.675005 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664889ee-8658-49b2-b0e0-a027ededa8ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:26:51.676457 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664889ee-8658-49b2-b0e0-a027ededa8ca service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:26:51.685688 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 734/1443] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:26:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:26:53.838052 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8573fc8c-f1a7-4e34-b174-539d8f3ff58d None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:53.838619 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8573fc8c-f1a7-4e34-b174-539d8f3ff58d None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:53.838619 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8573fc8c-f1a7-4e34-b174-539d8f3ff58d None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:53.840049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 710/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 7 15:26:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:26:53.845089 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf74e9-096a-47bd-a044-ca1a9a4b61bd None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:53.845516 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf74e9-096a-47bd-a044-ca1a9a4b61bd None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:53.845641 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf74e9-096a-47bd-a044-ca1a9a4b61bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:53.863917 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-f3bf74e9-096a-47bd-a044-ca1a9a4b61bd None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:26:53.865112 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-f3bf74e9-096a-47bd-a044-ca1a9a4b61bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:26:53.873439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 735/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 7 15:26:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:26:53.909778 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8573fc8c-f1a7-4e34-b174-539d8f3ff58d None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:26:53.921034 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8573fc8c-f1a7-4e34-b174-539d8f3ff58d None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:26:53.924470 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99bc2c-a060-4a89-951c-3fd03af853f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:26:53.925122 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99bc2c-a060-4a89-951c-3fd03af853f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:53.925399 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99bc2c-a060-4a89-951c-3fd03af853f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:53.925670 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99bc2c-a060-4a89-951c-3fd03af853f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:53.934993 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99bc2c-a060-4a89-951c-3fd03af853f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:26:53.936428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99bc2c-a060-4a89-951c-3fd03af853f5 service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:26:53.945539 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 711/1446] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:26:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:26:56.356425 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c5281-4381-4f7a-9c1a-2c9941822f6c None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:56.356969 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c5281-4381-4f7a-9c1a-2c9941822f6c None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:56.356969 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c5281-4381-4f7a-9c1a-2c9941822f6c None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:56.358584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 736/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 7 15:26:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:26:56.363974 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc27cdb-af9f-4c44-8b8d-35835a7f21ae None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:56.372391 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc27cdb-af9f-4c44-8b8d-35835a7f21ae None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:56.372391 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc27cdb-af9f-4c44-8b8d-35835a7f21ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:56.382663 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-2fc27cdb-af9f-4c44-8b8d-35835a7f21ae None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:26:56.384108 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-2fc27cdb-af9f-4c44-8b8d-35835a7f21ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:26:56.392041 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 712/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 7 15:26:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:26:56.430235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c5281-4381-4f7a-9c1a-2c9941822f6c None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:26:56.442574 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c5281-4381-4f7a-9c1a-2c9941822f6c None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:26:56.453441 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9f75f-5e60-4fc8-b660-66694722dbf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:26:56.454108 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9f75f-5e60-4fc8-b660-66694722dbf7 service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:56.454390 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9f75f-5e60-4fc8-b660-66694722dbf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:56.454664 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9f75f-5e60-4fc8-b660-66694722dbf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:56.464359 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f9f75f-5e60-4fc8-b660-66694722dbf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:26:56.466299 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f9f75f-5e60-4fc8-b660-66694722dbf7 service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:26:56.476448 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 737/1449] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:26:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:26:58.607048 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf0579-0b0e-4023-a5a9-f454236d67b7 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:58.607648 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf0579-0b0e-4023-a5a9-f454236d67b7 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:58.607648 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf0579-0b0e-4023-a5a9-f454236d67b7 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:58.609273 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 713/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 7 15:26:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:26:58.614492 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45bc23-aa10-46f4-b566-a63d9f873caa None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:58.614916 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45bc23-aa10-46f4-b566-a63d9f873caa None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:58.615059 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45bc23-aa10-46f4-b566-a63d9f873caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:58.632809 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-7f45bc23-aa10-46f4-b566-a63d9f873caa None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:26:58.634002 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-7f45bc23-aa10-46f4-b566-a63d9f873caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:26:58.642260 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 738/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 7 15:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:26:58.680826 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cf0579-0b0e-4023-a5a9-f454236d67b7 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:26:58.691703 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cf0579-0b0e-4023-a5a9-f454236d67b7 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:26:58.695185 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4703d-2850-44c7-89a3-7683d1db1f77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:26:58.695835 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4703d-2850-44c7-89a3-7683d1db1f77 service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:26:58.696114 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4703d-2850-44c7-89a3-7683d1db1f77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:26:58.696383 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4703d-2850-44c7-89a3-7683d1db1f77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:26:58.705648 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b4703d-2850-44c7-89a3-7683d1db1f77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:26:58.707050 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b4703d-2850-44c7-89a3-7683d1db1f77 service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:26:58.716473 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 714/1452] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:26:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:27:00.787480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd8a1d-e23e-4d0e-8c21-072425bf551a None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:00.787927 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd8a1d-e23e-4d0e-8c21-072425bf551a None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:00.788074 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd8a1d-e23e-4d0e-8c21-072425bf551a None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:00.789677 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 739/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 7 15:27:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:27:00.795187 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa2b11-0fca-418a-8c61-6a4c527202c6 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:00.795622 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa2b11-0fca-418a-8c61-6a4c527202c6 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:00.795784 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa2b11-0fca-418a-8c61-6a4c527202c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:00.814682 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-baaa2b11-0fca-418a-8c61-6a4c527202c6 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:27:00.815931 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-baaa2b11-0fca-418a-8c61-6a4c527202c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:27:00.824137 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 715/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 7 15:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:27:00.872981 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd8a1d-e23e-4d0e-8c21-072425bf551a None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:27:00.883692 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd8a1d-e23e-4d0e-8c21-072425bf551a None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:27:00.887155 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b315b5-0f92-4835-b8ca-277f865a02e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:27:00.887823 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b315b5-0f92-4835-b8ca-277f865a02e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:00.888114 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b315b5-0f92-4835-b8ca-277f865a02e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:00.888390 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b315b5-0f92-4835-b8ca-277f865a02e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:00.897234 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b315b5-0f92-4835-b8ca-277f865a02e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:27:00.898660 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b315b5-0f92-4835-b8ca-277f865a02e3 service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:27:00.907984 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 740/1455] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:27:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:27:03.039629 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9978332-2346-4768-a5f3-361393ae3940 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:03.040179 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9978332-2346-4768-a5f3-361393ae3940 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:03.040179 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9978332-2346-4768-a5f3-361393ae3940 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:03.041529 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 716/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 7 15:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:27:03.047297 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4f371-31a2-44c8-b69f-3c51f076e471 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:03.047739 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4f371-31a2-44c8-b69f-3c51f076e471 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:03.047877 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4f371-31a2-44c8-b69f-3c51f076e471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:03.066028 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-3fd4f371-31a2-44c8-b69f-3c51f076e471 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:27:03.067281 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-3fd4f371-31a2-44c8-b69f-3c51f076e471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:27:03.075111 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 741/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 7 15:27:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:27:03.113414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9978332-2346-4768-a5f3-361393ae3940 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:27:03.124518 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9978332-2346-4768-a5f3-361393ae3940 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:27:03.128016 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4732f48-ddfb-4d57-ad24-6d11c876cb1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:27:03.128775 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4732f48-ddfb-4d57-ad24-6d11c876cb1c service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:03.129054 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4732f48-ddfb-4d57-ad24-6d11c876cb1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:03.129323 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4732f48-ddfb-4d57-ad24-6d11c876cb1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:03.138913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4732f48-ddfb-4d57-ad24-6d11c876cb1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:27:03.140349 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4732f48-ddfb-4d57-ad24-6d11c876cb1c service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:27:03.149733 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 717/1458] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:27:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 15:27:05.247467 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5dddea-800b-46c9-9897-df7f55690c0e None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:05.248037 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5dddea-800b-46c9-9897-df7f55690c0e None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:05.248037 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5dddea-800b-46c9-9897-df7f55690c0e None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:05.249321 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 742/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 7 15:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:27:05.254640 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e50f9a-ac1f-4c30-99ca-09f15461a644 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:05.255106 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e50f9a-ac1f-4c30-99ca-09f15461a644 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:05.255198 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e50f9a-ac1f-4c30-99ca-09f15461a644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:05.273048 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-13e50f9a-ac1f-4c30-99ca-09f15461a644 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:27:05.274268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-13e50f9a-ac1f-4c30-99ca-09f15461a644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:27:05.282568 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 718/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 7 15:27:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:27:05.318710 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5dddea-800b-46c9-9897-df7f55690c0e None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:27:05.330043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5dddea-800b-46c9-9897-df7f55690c0e None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:27:05.333550 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbedd514-63fc-4751-ac83-87818a15f758 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:27:05.334202 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbedd514-63fc-4751-ac83-87818a15f758 service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:05.334485 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbedd514-63fc-4751-ac83-87818a15f758 service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:05.334759 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbedd514-63fc-4751-ac83-87818a15f758 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:05.344376 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbedd514-63fc-4751-ac83-87818a15f758 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:27:05.345813 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbedd514-63fc-4751-ac83-87818a15f758 service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:27:05.356249 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 743/1461] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:27:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:27:07.403198 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831bc4b-3c6a-4cee-9a32-b40dd25aa846 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:07.403782 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831bc4b-3c6a-4cee-9a32-b40dd25aa846 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:07.403881 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831bc4b-3c6a-4cee-9a32-b40dd25aa846 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:07.405188 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 719/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 7 15:27:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:27:07.410308 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5296964-9ab3-4f74-9e70-46ae899da5c3 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:07.410723 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5296964-9ab3-4f74-9e70-46ae899da5c3 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:07.410919 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5296964-9ab3-4f74-9e70-46ae899da5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:07.417145 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=79 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:27:07.429922 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-c5296964-9ab3-4f74-9e70-46ae899da5c3 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:27:07.431107 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-c5296964-9ab3-4f74-9e70-46ae899da5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:27:07.439601 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 744/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 7 15:27:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:27:07.475789 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831bc4b-3c6a-4cee-9a32-b40dd25aa846 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:27:07.481764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=73 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:27:07.487704 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831bc4b-3c6a-4cee-9a32-b40dd25aa846 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:27:07.491172 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f606fa3-49fb-4977-bfe2-72746ac14cde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:27:07.491823 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f606fa3-49fb-4977-bfe2-72746ac14cde service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:07.492109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f606fa3-49fb-4977-bfe2-72746ac14cde service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:07.492383 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f606fa3-49fb-4977-bfe2-72746ac14cde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:07.501725 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f606fa3-49fb-4977-bfe2-72746ac14cde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:27:07.503141 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f606fa3-49fb-4977-bfe2-72746ac14cde service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:27:07.512955 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 720/1464] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:27:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:27:09.527642 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db896051-471c-40f4-8411-dd1c87dda6a2 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:09.528189 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db896051-471c-40f4-8411-dd1c87dda6a2 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:09.528189 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db896051-471c-40f4-8411-dd1c87dda6a2 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:09.529528 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 745/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 7 15:27:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:27:09.534658 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73427c96-092c-4d90-a02d-5d8d7f947a66 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:09.535101 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73427c96-092c-4d90-a02d-5d8d7f947a66 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:09.535212 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73427c96-092c-4d90-a02d-5d8d7f947a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:09.555988 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-73427c96-092c-4d90-a02d-5d8d7f947a66 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:27:09.557258 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-73427c96-092c-4d90-a02d-5d8d7f947a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:27:09.565198 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 721/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 7 15:27:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:27:09.601151 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db896051-471c-40f4-8411-dd1c87dda6a2 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:27:09.611808 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db896051-471c-40f4-8411-dd1c87dda6a2 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:27:09.615285 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd877878-c91f-4df9-9908-d32320c8e7a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:27:09.615948 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd877878-c91f-4df9-9908-d32320c8e7a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:09.616225 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd877878-c91f-4df9-9908-d32320c8e7a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:09.616493 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd877878-c91f-4df9-9908-d32320c8e7a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:09.625391 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd877878-c91f-4df9-9908-d32320c8e7a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:27:09.626800 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd877878-c91f-4df9-9908-d32320c8e7a9 service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:27:09.636538 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 746/1467] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:27:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:27:11.677844 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1fbe53-fe37-4798-b3e9-947a20741f13 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:11.678356 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1fbe53-fe37-4798-b3e9-947a20741f13 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:11.678356 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1fbe53-fe37-4798-b3e9-947a20741f13 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:11.679727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 722/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 7 15:27:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:27:11.685481 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9d0aa-3d7f-4440-b0ef-66d884afdbf7 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:11.685880 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9d0aa-3d7f-4440-b0ef-66d884afdbf7 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:11.686030 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9d0aa-3d7f-4440-b0ef-66d884afdbf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:11.704310 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-60d9d0aa-3d7f-4440-b0ef-66d884afdbf7 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:27:11.705499 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-60d9d0aa-3d7f-4440-b0ef-66d884afdbf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:27:11.713426 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 747/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 7 15:27:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:27:11.749223 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1fbe53-fe37-4798-b3e9-947a20741f13 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:27:11.759950 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1fbe53-fe37-4798-b3e9-947a20741f13 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:27:11.764058 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd66c1dd-ecb0-4971-b13a-5141b2fe446a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:27:11.764689 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd66c1dd-ecb0-4971-b13a-5141b2fe446a service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:11.764969 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd66c1dd-ecb0-4971-b13a-5141b2fe446a service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:11.765246 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd66c1dd-ecb0-4971-b13a-5141b2fe446a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:11.774163 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd66c1dd-ecb0-4971-b13a-5141b2fe446a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:27:11.775596 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd66c1dd-ecb0-4971-b13a-5141b2fe446a service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:27:11.788241 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 723/1470] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:27:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:27:21.698704 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=11 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:27:21.780700 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=14 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:27:31.374476 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2a264-3804-4a84-af81-6fee529f048a None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:31.375112 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2a264-3804-4a84-af81-6fee529f048a None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:31.375112 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2a264-3804-4a84-af81-6fee529f048a None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:31.376417 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 748/1471] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:27:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:27:31.381743 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9364c5ca-6565-4977-be69-353efae761fa None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:31.382175 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9364c5ca-6565-4977-be69-353efae761fa None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:31.382438 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9364c5ca-6565-4977-be69-353efae761fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:31.400616 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-9364c5ca-6565-4977-be69-353efae761fa None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:27:31.402441 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-9364c5ca-6565-4977-be69-353efae761fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:27:31.410407 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 724/1472] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:27:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:27:31.495297 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2fecf-e9cf-4829-98b3-605b6d050894 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:31.495670 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2fecf-e9cf-4829-98b3-605b6d050894 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:31.495949 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2fecf-e9cf-4829-98b3-605b6d050894 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:31.497300 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 749/1473] 10.222.0.25 () {54 vars in 962 bytes} [Thu Jul 7 15:27:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:27:31.502231 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962cce4a-2e24-4bce-96dd-ad761e87d3c7 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:31.502524 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962cce4a-2e24-4bce-96dd-ad761e87d3c7 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:31.502836 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962cce4a-2e24-4bce-96dd-ad761e87d3c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:31.520994 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-962cce4a-2e24-4bce-96dd-ad761e87d3c7 None None] MFA Rules not processed for user `3d0d29e718f446e2a9fd1b2575a86041`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:27:31.522183 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-962cce4a-2e24-4bce-96dd-ad761e87d3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:27:31.529196 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 725/1474] 10.222.0.25 () {58 vars in 1059 bytes} [Thu Jul 7 15:27:31 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:27:31.536369 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2fecf-e9cf-4829-98b3-605b6d050894 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:27:31.547210 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2fecf-e9cf-4829-98b3-605b6d050894 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:27:31.550697 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d3521-7118-4a0d-8db7-04c45343a81f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0d29e718f446e2a9fd1b2575a86041', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:27:31.551316 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d3521-7118-4a0d-8db7-04c45343a81f service glance] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:31.551617 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d3521-7118-4a0d-8db7-04c45343a81f service glance] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:31.551893 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d3521-7118-4a0d-8db7-04c45343a81f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:31.560567 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9d3521-7118-4a0d-8db7-04c45343a81f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:27:31.561967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9d3521-7118-4a0d-8db7-04c45343a81f service glance] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:27:31.571127 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 750/1475] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:27:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:27:33.709532 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f0a4c-5a7f-4e62-bb83-c618d2540218 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:33.710041 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f0a4c-5a7f-4e62-bb83-c618d2540218 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:33.710041 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f0a4c-5a7f-4e62-bb83-c618d2540218 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:33.711411 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 726/1476] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:27:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:27:33.716538 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f282b-2812-4175-9981-cc2284a41c47 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:33.716932 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f282b-2812-4175-9981-cc2284a41c47 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:33.717087 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f282b-2812-4175-9981-cc2284a41c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:33.735491 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-8a3f282b-2812-4175-9981-cc2284a41c47 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:27:33.736697 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-8a3f282b-2812-4175-9981-cc2284a41c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:27:33.744431 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 751/1477] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:27:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:27:33.767287 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f0a4c-5a7f-4e62-bb83-c618d2540218 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:27:33.779886 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f0a4c-5a7f-4e62-bb83-c618d2540218 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:27:33.784446 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b4a608-4b92-4193-896b-5c463aaf1d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:27:33.785114 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4a608-4b92-4193-896b-5c463aaf1d24 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:33.785397 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4a608-4b92-4193-896b-5c463aaf1d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:33.785671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4a608-4b92-4193-896b-5c463aaf1d24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:33.786398 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b4a608-4b92-4193-896b-5c463aaf1d24 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:27:33.788198 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: WARNING py.warnings [None req-49b4a608-4b92-4193-896b-5c463aaf1d24 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 07 15:27:33.788198 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: warnings.warn(msg) Jul 07 15:27:33.788198 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: Jul 07 15:27:33.789363 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b4a608-4b92-4193-896b-5c463aaf1d24 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:27:33.813041 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 727/1478] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Jul 7 15:27:33 2022] GET /identity/v3/projects => generated 2553 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:27:35.879968 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16440b6-ce57-4a9e-8db4-fbb27a692041 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:35.880535 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16440b6-ce57-4a9e-8db4-fbb27a692041 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:35.880535 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16440b6-ce57-4a9e-8db4-fbb27a692041 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:35.881884 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 752/1479] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:27:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:27:35.887139 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0098e-689e-40cf-848a-26bba247e6a6 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:35.887591 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0098e-689e-40cf-848a-26bba247e6a6 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:35.887736 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0098e-689e-40cf-848a-26bba247e6a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:35.905226 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-fae0098e-689e-40cf-848a-26bba247e6a6 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:27:35.906435 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-fae0098e-689e-40cf-848a-26bba247e6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:27:35.914216 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 728/1480] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:27:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:27:35.956841 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16440b6-ce57-4a9e-8db4-fbb27a692041 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:27:35.971315 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16440b6-ce57-4a9e-8db4-fbb27a692041 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:27:35.976695 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deecb757-832d-4748-ba78-36cb33f86746 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:27:35.977329 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deecb757-832d-4748-ba78-36cb33f86746 service nova] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:35.977611 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deecb757-832d-4748-ba78-36cb33f86746 service nova] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:35.977886 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deecb757-832d-4748-ba78-36cb33f86746 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:35.986664 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deecb757-832d-4748-ba78-36cb33f86746 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:27:35.988267 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deecb757-832d-4748-ba78-36cb33f86746 service nova] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:27:35.998208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 753/1481] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:27:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:27:38.182185 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac55b6-334e-4ab4-b166-c340f0858816 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:38.182721 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac55b6-334e-4ab4-b166-c340f0858816 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:38.182721 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac55b6-334e-4ab4-b166-c340f0858816 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:38.184161 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 729/1482] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:27:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:27:38.189210 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535d6a3-2773-4b3d-98a5-f3454a12f2e6 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:38.189593 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535d6a3-2773-4b3d-98a5-f3454a12f2e6 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:38.189762 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535d6a3-2773-4b3d-98a5-f3454a12f2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:38.207679 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-e535d6a3-2773-4b3d-98a5-f3454a12f2e6 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:27:38.208867 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-e535d6a3-2773-4b3d-98a5-f3454a12f2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:27:38.217126 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 754/1483] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:27:38.251825 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac55b6-334e-4ab4-b166-c340f0858816 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:27:38.262666 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac55b6-334e-4ab4-b166-c340f0858816 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:27:38.266075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6079e-257d-4cb8-bd20-5d35715ebf53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:27:38.266698 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6079e-257d-4cb8-bd20-5d35715ebf53 service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:38.266976 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6079e-257d-4cb8-bd20-5d35715ebf53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:38.267270 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6079e-257d-4cb8-bd20-5d35715ebf53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:38.275953 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd6079e-257d-4cb8-bd20-5d35715ebf53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:27:38.277968 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd6079e-257d-4cb8-bd20-5d35715ebf53 service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:27:38.287082 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 730/1484] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:27:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:27:40.351802 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455f2e3-5e05-4ee9-8002-2f46543c090b None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:40.352351 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455f2e3-5e05-4ee9-8002-2f46543c090b None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:40.352351 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455f2e3-5e05-4ee9-8002-2f46543c090b None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:40.353711 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 755/1485] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:27:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:27:40.359184 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61680e44-6d7e-476f-a52d-0b8434a66181 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:40.359676 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61680e44-6d7e-476f-a52d-0b8434a66181 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:40.359811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61680e44-6d7e-476f-a52d-0b8434a66181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:40.377439 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-61680e44-6d7e-476f-a52d-0b8434a66181 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:27:40.378628 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-61680e44-6d7e-476f-a52d-0b8434a66181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:27:40.386943 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 731/1486] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:27:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:27:40.424243 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455f2e3-5e05-4ee9-8002-2f46543c090b None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:27:40.434797 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455f2e3-5e05-4ee9-8002-2f46543c090b None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:27:40.438234 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaaaa2c-86b7-4fad-b146-bbd47f6e3cfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:27:40.438859 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaaaa2c-86b7-4fad-b146-bbd47f6e3cfd service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:40.439136 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaaaa2c-86b7-4fad-b146-bbd47f6e3cfd service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:40.439408 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaaaa2c-86b7-4fad-b146-bbd47f6e3cfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:40.448065 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaaaa2c-86b7-4fad-b146-bbd47f6e3cfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:27:40.449966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaaaa2c-86b7-4fad-b146-bbd47f6e3cfd service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:27:40.459075 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 756/1487] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:27:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:27:42.629698 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4daae0-8c15-47e6-80f9-82dcf6d7ca12 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:42.630236 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4daae0-8c15-47e6-80f9-82dcf6d7ca12 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:42.630236 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4daae0-8c15-47e6-80f9-82dcf6d7ca12 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:42.631676 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 732/1488] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:27:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:27:42.637381 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e11f3-9d84-4d4d-87b3-13abfa5735be None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:42.637795 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e11f3-9d84-4d4d-87b3-13abfa5735be None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:42.637947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e11f3-9d84-4d4d-87b3-13abfa5735be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:42.655836 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-d87e11f3-9d84-4d4d-87b3-13abfa5735be None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:27:42.657010 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-d87e11f3-9d84-4d4d-87b3-13abfa5735be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:27:42.664925 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 757/1489] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:27:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:27:42.704238 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4daae0-8c15-47e6-80f9-82dcf6d7ca12 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:27:42.715321 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4daae0-8c15-47e6-80f9-82dcf6d7ca12 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:27:42.718956 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb3008-d8a2-4322-9b2f-69a43ac2f71a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:27:42.719613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb3008-d8a2-4322-9b2f-69a43ac2f71a service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:42.719898 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb3008-d8a2-4322-9b2f-69a43ac2f71a service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:42.720175 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb3008-d8a2-4322-9b2f-69a43ac2f71a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:42.729011 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cb3008-d8a2-4322-9b2f-69a43ac2f71a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:27:42.730417 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cb3008-d8a2-4322-9b2f-69a43ac2f71a service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:27:42.739968 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 733/1490] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:27:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:27:45.093470 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753655f-179e-4766-a3cc-51eb3d7f0036 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:45.094014 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753655f-179e-4766-a3cc-51eb3d7f0036 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:45.094014 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753655f-179e-4766-a3cc-51eb3d7f0036 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:45.095379 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 758/1491] 10.222.0.25 () {54 vars in 926 bytes} [Thu Jul 7 15:27:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:27:45.100447 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662fc584-c2d7-411e-9ecc-2a4cb8d1c0be None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:45.100857 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662fc584-c2d7-411e-9ecc-2a4cb8d1c0be None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:45.100996 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662fc584-c2d7-411e-9ecc-2a4cb8d1c0be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:45.118591 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-662fc584-c2d7-411e-9ecc-2a4cb8d1c0be None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:27:45.119794 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-662fc584-c2d7-411e-9ecc-2a4cb8d1c0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:27:45.127435 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 734/1492] 10.222.0.25 () {58 vars in 1023 bytes} [Thu Jul 7 15:27:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:27:45.166430 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2753655f-179e-4766-a3cc-51eb3d7f0036 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:27:45.176893 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2753655f-179e-4766-a3cc-51eb3d7f0036 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:27:45.180443 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ececd3-40c5-43c2-a2ea-52d22911a206 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:27:45.181099 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ececd3-40c5-43c2-a2ea-52d22911a206 service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:27:45.181457 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ececd3-40c5-43c2-a2ea-52d22911a206 service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:27:45.181657 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ececd3-40c5-43c2-a2ea-52d22911a206 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:27:45.191152 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ececd3-40c5-43c2-a2ea-52d22911a206 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:27:45.192601 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ececd3-40c5-43c2-a2ea-52d22911a206 service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:27:45.202664 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 759/1493] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:27:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:27:55.113742 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=33 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:27:55.197834 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=33 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:29:19.718361 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478edf7a-a09a-401b-9ea5-2c31dead9ade None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:29:19.718951 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478edf7a-a09a-401b-9ea5-2c31dead9ade None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:29:19.718951 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478edf7a-a09a-401b-9ea5-2c31dead9ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:29:19.736950 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-478edf7a-a09a-401b-9ea5-2c31dead9ade None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:29:19.738132 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-478edf7a-a09a-401b-9ea5-2c31dead9ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:29:19.746617 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 735/1494] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:29:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:29:19.754288 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421358a-c7bb-49a5-b0b9-f417125aa0ba None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:29:19.754760 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421358a-c7bb-49a5-b0b9-f417125aa0ba None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:29:19.754867 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421358a-c7bb-49a5-b0b9-f417125aa0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:29:19.775749 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-f421358a-c7bb-49a5-b0b9-f417125aa0ba None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:29:19.777554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-f421358a-c7bb-49a5-b0b9-f417125aa0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:29:19.786761 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 760/1495] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:29:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:29:19.791420 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478edf7a-a09a-401b-9ea5-2c31dead9ade None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:29:19.801004 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478edf7a-a09a-401b-9ea5-2c31dead9ade None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:29:19.806009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1672de06-ea1e-4a51-bd31-a8b79243fb09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:29:19.806653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672de06-ea1e-4a51-bd31-a8b79243fb09 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:29:19.806935 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672de06-ea1e-4a51-bd31-a8b79243fb09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:29:19.807211 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672de06-ea1e-4a51-bd31-a8b79243fb09 admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:29:19.807976 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1672de06-ea1e-4a51-bd31-a8b79243fb09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:29:19.808284 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1672de06-ea1e-4a51-bd31-a8b79243fb09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:29:19.809274 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: WARNING py.warnings [None req-1672de06-ea1e-4a51-bd31-a8b79243fb09 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 07 15:29:19.809274 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: warnings.warn(msg) Jul 07 15:29:19.809274 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: Jul 07 15:29:19.810160 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1672de06-ea1e-4a51-bd31-a8b79243fb09 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:29:19.822144 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 736/1496] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:29:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:29:19.829918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f421358a-c7bb-49a5-b0b9-f417125aa0ba None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:29:19.838785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f421358a-c7bb-49a5-b0b9-f417125aa0ba None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:29:19.843194 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3fc39e-fcc6-47c6-9b72-64e2dc819613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:29:19.843863 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fc39e-fcc6-47c6-9b72-64e2dc819613 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:29:19.844142 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fc39e-fcc6-47c6-9b72-64e2dc819613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:29:19.844410 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fc39e-fcc6-47c6-9b72-64e2dc819613 admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:29:19.845083 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3fc39e-fcc6-47c6-9b72-64e2dc819613 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:29:19.846784 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: WARNING py.warnings [None req-4a3fc39e-fcc6-47c6-9b72-64e2dc819613 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 07 15:29:19.846784 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: warnings.warn(msg) Jul 07 15:29:19.846784 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: Jul 07 15:29:19.848009 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3fc39e-fcc6-47c6-9b72-64e2dc819613 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:29:19.971181 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 761/1497] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:29:19 2022] POST /identity/v3/projects => generated 377 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:29:19.975071 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1672de06-ea1e-4a51-bd31-a8b79243fb09 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:29:19.985282 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1672de06-ea1e-4a51-bd31-a8b79243fb09 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:29:20.041116 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282b1f4-6bb3-40ee-98ea-5dc25fdd1f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:29:20.041772 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282b1f4-6bb3-40ee-98ea-5dc25fdd1f63 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:29:20.042050 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282b1f4-6bb3-40ee-98ea-5dc25fdd1f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:29:20.042323 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282b1f4-6bb3-40ee-98ea-5dc25fdd1f63 admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:29:20.043020 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3282b1f4-6bb3-40ee-98ea-5dc25fdd1f63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:29:20.044398 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: WARNING py.warnings [None req-3282b1f4-6bb3-40ee-98ea-5dc25fdd1f63 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 07 15:29:20.044398 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: warnings.warn(msg) Jul 07 15:29:20.044398 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: Jul 07 15:29:20.045944 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3282b1f4-6bb3-40ee-98ea-5dc25fdd1f63 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:29:20.152785 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 737/1498] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:29:19 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:29:20.158588 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3fc39e-fcc6-47c6-9b72-64e2dc819613 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:29:20.169567 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3fc39e-fcc6-47c6-9b72-64e2dc819613 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:29:20.174102 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f09c854-f082-4a7a-8f9b-6d1729175a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:29:20.174757 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f09c854-f082-4a7a-8f9b-6d1729175a78 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:29:20.175042 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f09c854-f082-4a7a-8f9b-6d1729175a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:29:20.175318 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f09c854-f082-4a7a-8f9b-6d1729175a78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:29:20.176085 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f09c854-f082-4a7a-8f9b-6d1729175a78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:29:20.177085 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: WARNING py.warnings [None req-0f09c854-f082-4a7a-8f9b-6d1729175a78 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 07 15:29:20.177085 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: warnings.warn(msg) Jul 07 15:29:20.177085 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: Jul 07 15:29:20.178552 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f09c854-f082-4a7a-8f9b-6d1729175a78 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:29:20.191067 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 762/1499] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:29:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:29:20.194940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282b1f4-6bb3-40ee-98ea-5dc25fdd1f63 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:29:20.204677 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282b1f4-6bb3-40ee-98ea-5dc25fdd1f63 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:29:20.209139 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d9e63-9da1-45e5-90b6-78fcde5f289e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:29:20.209790 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d9e63-9da1-45e5-90b6-78fcde5f289e admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:29:20.210077 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d9e63-9da1-45e5-90b6-78fcde5f289e admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:29:20.210353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d9e63-9da1-45e5-90b6-78fcde5f289e admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:29:20.211072 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d9e63-9da1-45e5-90b6-78fcde5f289e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:29:20.212099 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: WARNING py.warnings [None req-4f7d9e63-9da1-45e5-90b6-78fcde5f289e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 07 15:29:20.212099 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: warnings.warn(msg) Jul 07 15:29:20.212099 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: Jul 07 15:29:20.212969 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d9e63-9da1-45e5-90b6-78fcde5f289e admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:29:20.227510 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 738/1500] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:29:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:29:20.231383 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f09c854-f082-4a7a-8f9b-6d1729175a78 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:29:20.242134 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f09c854-f082-4a7a-8f9b-6d1729175a78 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:29:20.246591 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add54674-cac8-4b24-9372-6eb688fc915c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:29:20.247465 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add54674-cac8-4b24-9372-6eb688fc915c admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:29:20.247854 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add54674-cac8-4b24-9372-6eb688fc915c admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:29:20.248134 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add54674-cac8-4b24-9372-6eb688fc915c admin admin] PATH_INFO: `/v3/projects/c547ac8e60b8450abdac765fe8a58659/users/cff6cec9beab410284d7c64d9f0edd8a/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:29:20.262899 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add54674-cac8-4b24-9372-6eb688fc915c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c547ac8e60b8450abdac765fe8a58659, user_id=cff6cec9beab410284d7c64d9f0edd8a, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:29:20.263973 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: WARNING py.warnings [None req-add54674-cac8-4b24-9372-6eb688fc915c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 07 15:29:20.263973 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: warnings.warn(msg) Jul 07 15:29:20.263973 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: Jul 07 15:29:20.265903 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add54674-cac8-4b24-9372-6eb688fc915c admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:29:20.340948 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 763/1501] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:29:20 2022] PUT /identity/v3/projects/c547ac8e60b8450abdac765fe8a58659/users/cff6cec9beab410284d7c64d9f0edd8a/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:29:20.353748 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02116eae-a322-42bc-af7f-72050ec71c96 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:29:20.354031 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02116eae-a322-42bc-af7f-72050ec71c96 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:29:20.354307 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02116eae-a322-42bc-af7f-72050ec71c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:29:20.380487 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-02116eae-a322-42bc-af7f-72050ec71c96 None None] MFA Rules not processed for user `cff6cec9beab410284d7c64d9f0edd8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:29:20.381740 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-02116eae-a322-42bc-af7f-72050ec71c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:29:20.457697 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 739/1502] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:29:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:29:20.471293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733fdec-e1d1-47f0-9f8d-2dfa2afcec11 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:29:20.473792 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733fdec-e1d1-47f0-9f8d-2dfa2afcec11 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:29:20.473792 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733fdec-e1d1-47f0-9f8d-2dfa2afcec11 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:29:20.473792 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 764/1503] 10.222.0.25 () {54 vars in 963 bytes} [Thu Jul 7 15:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:29:20.478576 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9293d8-8801-4319-8ffb-5d5227d3bb33 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:29:20.478813 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9293d8-8801-4319-8ffb-5d5227d3bb33 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:29:20.479085 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9293d8-8801-4319-8ffb-5d5227d3bb33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:29:20.496810 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-7e9293d8-8801-4319-8ffb-5d5227d3bb33 None None] MFA Rules not processed for user `2a0fbebb3dd2458ba4fe5d7f0ce211b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:29:20.497988 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-7e9293d8-8801-4319-8ffb-5d5227d3bb33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:29:20.540277 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:29:20.547423 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 740/1504] 10.222.0.25 () {58 vars in 1060 bytes} [Thu Jul 7 15:29:20 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:29:20.554994 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e733fdec-e1d1-47f0-9f8d-2dfa2afcec11 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:29:20.565676 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e733fdec-e1d1-47f0-9f8d-2dfa2afcec11 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:29:20.569243 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a874b2-d4a4-4855-a865-214597435c14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:29:20.569881 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a874b2-d4a4-4855-a865-214597435c14 service cinder] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:29:20.570166 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a874b2-d4a4-4855-a865-214597435c14 service cinder] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:29:20.570442 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a874b2-d4a4-4855-a865-214597435c14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:29:20.579277 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a874b2-d4a4-4855-a865-214597435c14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:29:20.582547 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a874b2-d4a4-4855-a865-214597435c14 service cinder] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:29:20.590124 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 765/1505] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:29:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 15:29:20.639146 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf49553-49b4-45be-a792-c33b009b545f None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:29:20.639427 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf49553-49b4-45be-a792-c33b009b545f None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:29:20.639746 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf49553-49b4-45be-a792-c33b009b545f None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:29:20.641027 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 741/1506] 10.222.0.25 () {52 vars in 829 bytes} [Thu Jul 7 15:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:29:20.644589 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a874b2-d4a4-4855-a865-214597435c14 service cinder] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:29:20.706106 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a874b2-d4a4-4855-a865-214597435c14 service cinder] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:29:20.710586 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25658d85-2411-45ea-bae0-34090a9f6b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:29:20.711446 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25658d85-2411-45ea-bae0-34090a9f6b93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:29:20.711758 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25658d85-2411-45ea-bae0-34090a9f6b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:29:20.712031 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25658d85-2411-45ea-bae0-34090a9f6b93 admin admin] PATH_INFO: `/v3/users/cff6cec9beab410284d7c64d9f0edd8a` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:29:20.717764 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25658d85-2411-45ea-bae0-34090a9f6b93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cff6cec9beab410284d7c64d9f0edd8a)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:29:20.718816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: WARNING py.warnings [None req-25658d85-2411-45ea-bae0-34090a9f6b93 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 07 15:29:20.718816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: warnings.warn(msg) Jul 07 15:29:20.718816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: Jul 07 15:29:20.720076 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25658d85-2411-45ea-bae0-34090a9f6b93 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:29:20.959930 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-25658d85-2411-45ea-bae0-34090a9f6b93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cff6cec9beab410284d7c64d9f0edd8a'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:29:20.960591 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-25658d85-2411-45ea-bae0-34090a9f6b93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cff6cec9beab410284d7c64d9f0edd8a'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:29:20.974045 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-25658d85-2411-45ea-bae0-34090a9f6b93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cff6cec9beab410284d7c64d9f0edd8a'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:29:20.974860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-25658d85-2411-45ea-bae0-34090a9f6b93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cff6cec9beab410284d7c64d9f0edd8a'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:29:20.994800 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-25658d85-2411-45ea-bae0-34090a9f6b93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cff6cec9beab410284d7c64d9f0edd8a'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:29:20.998721 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:29:21.072069 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 766/1507] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:29:20 2022] DELETE /identity/v3/users/cff6cec9beab410284d7c64d9f0edd8a => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:29:21.076451 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b74cb-c16f-4645-85e7-8dcecf273018 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:29:21.076934 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b74cb-c16f-4645-85e7-8dcecf273018 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:29:21.077043 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b74cb-c16f-4645-85e7-8dcecf273018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:29:21.094954 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-051b74cb-c16f-4645-85e7-8dcecf273018 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:29:21.096187 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-051b74cb-c16f-4645-85e7-8dcecf273018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:29:21.155486 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 742/1508] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:29:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:29:21.165088 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25658d85-2411-45ea-bae0-34090a9f6b93 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:29:21.166511 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-25658d85-2411-45ea-bae0-34090a9f6b93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:29:21.224064 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25658d85-2411-45ea-bae0-34090a9f6b93 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:29:21.227674 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa8b69-8c21-4dd0-b204-925d67fe21d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:29:21.228322 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa8b69-8c21-4dd0-b204-925d67fe21d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:29:21.228606 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa8b69-8c21-4dd0-b204-925d67fe21d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:29:21.228887 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa8b69-8c21-4dd0-b204-925d67fe21d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:29:21.237954 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aa8b69-8c21-4dd0-b204-925d67fe21d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:29:21.239383 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aa8b69-8c21-4dd0-b204-925d67fe21d2 service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:29:21.249338 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 767/1509] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:29:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:29:21.756015 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051b74cb-c16f-4645-85e7-8dcecf273018 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:29:21.757566 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-051b74cb-c16f-4645-85e7-8dcecf273018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:29:21.769645 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051b74cb-c16f-4645-85e7-8dcecf273018 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:29:21.774681 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06077b69-9eae-4405-b04f-836ef358123f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:29:21.775499 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06077b69-9eae-4405-b04f-836ef358123f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:29:21.775811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06077b69-9eae-4405-b04f-836ef358123f admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:29:21.776087 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06077b69-9eae-4405-b04f-836ef358123f admin admin] PATH_INFO: `/v3/projects/c547ac8e60b8450abdac765fe8a58659` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:29:21.777331 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06077b69-9eae-4405-b04f-836ef358123f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c547ac8e60b8450abdac765fe8a58659)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:29:21.778351 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: WARNING py.warnings [None req-06077b69-9eae-4405-b04f-836ef358123f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 07 15:29:21.778351 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: warnings.warn(msg) Jul 07 15:29:21.778351 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: Jul 07 15:29:21.779559 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06077b69-9eae-4405-b04f-836ef358123f admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:29:21.892334 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-06077b69-9eae-4405-b04f-836ef358123f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c547ac8e60b8450abdac765fe8a58659', 'request_id': 'req-06077b69-9eae-4405-b04f-836ef358123f'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:29:21.898829 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-06077b69-9eae-4405-b04f-836ef358123f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c547ac8e60b8450abdac765fe8a58659', 'request_id': 'req-06077b69-9eae-4405-b04f-836ef358123f'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:29:21.911970 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-06077b69-9eae-4405-b04f-836ef358123f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c547ac8e60b8450abdac765fe8a58659', 'request_id': 'req-06077b69-9eae-4405-b04f-836ef358123f'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:29:21.995648 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-06077b69-9eae-4405-b04f-836ef358123f admin admin] The token cache is being invalidate because project c547ac8e60b8450abdac765fe8a58659 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:29:21.995940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-06077b69-9eae-4405-b04f-836ef358123f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:29:21.996655 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-06077b69-9eae-4405-b04f-836ef358123f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:29:21.998236 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 743/1510] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:29:21 2022] DELETE /identity/v3/projects/c547ac8e60b8450abdac765fe8a58659 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:29:31.245182 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:29:31.992382 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:30:57.393754 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89fe4d2-0964-4191-ae88-086b87316920 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:30:57.394372 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89fe4d2-0964-4191-ae88-086b87316920 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:30:57.394372 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89fe4d2-0964-4191-ae88-086b87316920 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:30:57.396199 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 768/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 7 15:30:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:30:57.400935 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45bd5d-ae12-4cc4-bb50-7b2a38a1a5b8 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:30:57.401371 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45bd5d-ae12-4cc4-bb50-7b2a38a1a5b8 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:30:57.401479 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45bd5d-ae12-4cc4-bb50-7b2a38a1a5b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:30:57.420234 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-9b45bd5d-ae12-4cc4-bb50-7b2a38a1a5b8 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:30:57.421498 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-9b45bd5d-ae12-4cc4-bb50-7b2a38a1a5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:30:57.479803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 744/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 7 15:30:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:30:57.631920 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89fe4d2-0964-4191-ae88-086b87316920 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:30:57.633606 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-c89fe4d2-0964-4191-ae88-086b87316920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:30:57.706838 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89fe4d2-0964-4191-ae88-086b87316920 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:30:57.712740 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f6578-a265-477e-ac87-8dc4e8556367 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:30:57.713373 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f6578-a265-477e-ac87-8dc4e8556367 service nova] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:30:57.713654 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f6578-a265-477e-ac87-8dc4e8556367 service nova] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:30:57.713925 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f6578-a265-477e-ac87-8dc4e8556367 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:30:57.721849 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8f6578-a265-477e-ac87-8dc4e8556367 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:30:57.723414 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8f6578-a265-477e-ac87-8dc4e8556367 service nova] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:30:57.733092 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 769/1513] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:30:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:30:58.555726 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a1f56-8ba1-404e-9eae-be38650d210b None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:30:58.556251 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a1f56-8ba1-404e-9eae-be38650d210b None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:30:58.556251 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a1f56-8ba1-404e-9eae-be38650d210b None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:30:58.557621 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 745/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 7 15:30:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:30:58.562594 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c292596-3a3b-4f48-bc2b-6eb3f10e1725 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:30:58.562988 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c292596-3a3b-4f48-bc2b-6eb3f10e1725 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:30:58.563155 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c292596-3a3b-4f48-bc2b-6eb3f10e1725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:30:58.581488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-7c292596-3a3b-4f48-bc2b-6eb3f10e1725 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:30:58.583194 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-7c292596-3a3b-4f48-bc2b-6eb3f10e1725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:30:58.591188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 770/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 7 15:30:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:30:58.743443 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a1f56-8ba1-404e-9eae-be38650d210b None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:30:58.757721 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a1f56-8ba1-404e-9eae-be38650d210b None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:30:58.763035 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d8368-74ca-43e7-a596-ddbfaccc5a38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:30:58.763709 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d8368-74ca-43e7-a596-ddbfaccc5a38 service nova] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:30:58.763996 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d8368-74ca-43e7-a596-ddbfaccc5a38 service nova] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:30:58.764276 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d8368-74ca-43e7-a596-ddbfaccc5a38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:30:58.772883 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5d8368-74ca-43e7-a596-ddbfaccc5a38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:30:58.774460 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5d8368-74ca-43e7-a596-ddbfaccc5a38 service nova] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:30:58.785039 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 746/1516] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:30:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:30:59.405043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a8b931-9d48-49f8-bf43-30b88dfb0fdc None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:30:59.405328 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a8b931-9d48-49f8-bf43-30b88dfb0fdc None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:30:59.405627 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a8b931-9d48-49f8-bf43-30b88dfb0fdc None None] PATH_INFO: `/v3` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:30:59.406915 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 771/1517] 10.222.0.50 () {54 vars in 925 bytes} [Thu Jul 7 15:30:59 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:30:59.417947 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a30fa2-5632-4d78-b9e7-ff8fd90f3a7c None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:30:59.418237 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a30fa2-5632-4d78-b9e7-ff8fd90f3a7c None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:30:59.418510 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a30fa2-5632-4d78-b9e7-ff8fd90f3a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:30:59.439155 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-47a30fa2-5632-4d78-b9e7-ff8fd90f3a7c None None] MFA Rules not processed for user `755c2782cfb042f89950b8fc862e4fa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:30:59.440374 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-47a30fa2-5632-4d78-b9e7-ff8fd90f3a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:30:59.619838 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 747/1518] 10.222.0.50 () {58 vars in 1015 bytes} [Thu Jul 7 15:30:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:31:03.602467 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a25c3-c58f-46e5-9af1-564a88cd48a2 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:31:03.602978 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a25c3-c58f-46e5-9af1-564a88cd48a2 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:31:03.602978 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a25c3-c58f-46e5-9af1-564a88cd48a2 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:31:03.604519 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 772/1519] 10.222.0.25 () {54 vars in 960 bytes} [Thu Jul 7 15:31:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:31:03.613149 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fd3e6-4fae-4398-8580-9a7684a31abb None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:31:03.613149 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fd3e6-4fae-4398-8580-9a7684a31abb None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:31:03.613149 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fd3e6-4fae-4398-8580-9a7684a31abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:31:03.634144 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-200fd3e6-4fae-4398-8580-9a7684a31abb None None] MFA Rules not processed for user `755c2782cfb042f89950b8fc862e4fa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:31:03.635325 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-200fd3e6-4fae-4398-8580-9a7684a31abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:31:03.643924 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 748/1520] 10.222.0.25 () {58 vars in 1057 bytes} [Thu Jul 7 15:31:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:31:03.652056 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a25c3-c58f-46e5-9af1-564a88cd48a2 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:31:03.665161 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a25c3-c58f-46e5-9af1-564a88cd48a2 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:31:03.670086 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a06d047-f9d6-4727-8f28-5b8492ac424c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '755c2782cfb042f89950b8fc862e4fa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:31:03.670716 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06d047-f9d6-4727-8f28-5b8492ac424c service placement] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:31:03.671028 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06d047-f9d6-4727-8f28-5b8492ac424c service placement] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:31:03.671301 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06d047-f9d6-4727-8f28-5b8492ac424c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:31:03.679484 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a06d047-f9d6-4727-8f28-5b8492ac424c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:31:03.681087 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a06d047-f9d6-4727-8f28-5b8492ac424c service placement] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:31:03.690592 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 773/1521] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:31:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:31:10.027809 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e0ac6-6924-4856-afee-fbb9cb702826 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:31:10.028327 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e0ac6-6924-4856-afee-fbb9cb702826 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:31:10.028437 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e0ac6-6924-4856-afee-fbb9cb702826 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:31:10.029705 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 749/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 7 15:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:31:10.034418 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85df6ec-65ff-4516-a718-5c61ccc56162 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:31:10.034883 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85df6ec-65ff-4516-a718-5c61ccc56162 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:31:10.035629 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85df6ec-65ff-4516-a718-5c61ccc56162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:31:10.053420 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-e85df6ec-65ff-4516-a718-5c61ccc56162 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:31:10.054616 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-e85df6ec-65ff-4516-a718-5c61ccc56162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:31:10.062349 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 774/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 7 15:31:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:31:10.220214 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942e0ac6-6924-4856-afee-fbb9cb702826 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:31:10.236414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942e0ac6-6924-4856-afee-fbb9cb702826 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:31:10.242251 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc600d-9d21-4ba2-a105-d5528f3879cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:31:10.242885 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc600d-9d21-4ba2-a105-d5528f3879cf service nova] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:31:10.243164 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc600d-9d21-4ba2-a105-d5528f3879cf service nova] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:31:10.243437 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc600d-9d21-4ba2-a105-d5528f3879cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:31:10.251999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc600d-9d21-4ba2-a105-d5528f3879cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:31:10.256755 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc600d-9d21-4ba2-a105-d5528f3879cf service nova] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:31:10.264275 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 750/1524] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:31:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:31:11.088182 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40f06d-8d81-4eae-a1fd-1d8755d02684 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:31:11.088737 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40f06d-8d81-4eae-a1fd-1d8755d02684 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:31:11.088737 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40f06d-8d81-4eae-a1fd-1d8755d02684 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:31:11.090113 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 775/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 7 15:31:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:31:11.095112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ed1b5-d9aa-46c7-8b94-d8b45f1b98bc None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:31:11.095538 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ed1b5-d9aa-46c7-8b94-d8b45f1b98bc None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:31:11.095734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ed1b5-d9aa-46c7-8b94-d8b45f1b98bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:31:11.114456 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-074ed1b5-d9aa-46c7-8b94-d8b45f1b98bc None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:31:11.115667 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-074ed1b5-d9aa-46c7-8b94-d8b45f1b98bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:31:11.123229 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 751/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 7 15:31:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:31:11.273429 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40f06d-8d81-4eae-a1fd-1d8755d02684 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:31:11.287538 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40f06d-8d81-4eae-a1fd-1d8755d02684 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:31:11.292858 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c78005f-0484-4eab-843f-adc912a2d512 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:31:11.293487 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78005f-0484-4eab-843f-adc912a2d512 service nova] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:31:11.293770 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78005f-0484-4eab-843f-adc912a2d512 service nova] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:31:11.294045 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78005f-0484-4eab-843f-adc912a2d512 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:31:11.302659 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c78005f-0484-4eab-843f-adc912a2d512 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:31:11.304313 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c78005f-0484-4eab-843f-adc912a2d512 service nova] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:31:11.314227 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 776/1527] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:31:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:31:14.946746 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3a741-5fd6-4229-a134-c0d59b98ca9e None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:31:14.947316 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3a741-5fd6-4229-a134-c0d59b98ca9e None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:31:14.947316 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3a741-5fd6-4229-a134-c0d59b98ca9e None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:31:14.948770 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 752/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 7 15:31:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:31:14.953876 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89b2aa-fa3a-4239-8ade-614c01bf0197 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:31:14.954323 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89b2aa-fa3a-4239-8ade-614c01bf0197 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:31:14.954439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89b2aa-fa3a-4239-8ade-614c01bf0197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:31:14.973069 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-fb89b2aa-fa3a-4239-8ade-614c01bf0197 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:31:14.974260 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-fb89b2aa-fa3a-4239-8ade-614c01bf0197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:31:14.982300 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 777/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 7 15:31:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:31:15.054277 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3a741-5fd6-4229-a134-c0d59b98ca9e None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:31:15.056006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-69e3a741-5fd6-4229-a134-c0d59b98ca9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:31:15.122187 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3a741-5fd6-4229-a134-c0d59b98ca9e None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:31:15.127032 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d38f0-bc00-4132-afd5-0b6907e4218b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:31:15.127710 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d38f0-bc00-4132-afd5-0b6907e4218b service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:31:15.127998 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d38f0-bc00-4132-afd5-0b6907e4218b service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:31:15.128277 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d38f0-bc00-4132-afd5-0b6907e4218b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:31:15.137402 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279d38f0-bc00-4132-afd5-0b6907e4218b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:31:15.138826 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279d38f0-bc00-4132-afd5-0b6907e4218b service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:31:15.143790 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:31:15.149272 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 753/1530] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:31:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:31:17.208583 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb435f-1808-4997-bc68-1882162ac93e None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:31:17.209231 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb435f-1808-4997-bc68-1882162ac93e None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:31:17.209231 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb435f-1808-4997-bc68-1882162ac93e None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:31:17.210654 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 778/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 7 15:31:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:31:17.216509 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6975ce4-bacb-4ef7-a88e-c10db58488f8 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:31:17.217047 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6975ce4-bacb-4ef7-a88e-c10db58488f8 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:31:17.217172 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6975ce4-bacb-4ef7-a88e-c10db58488f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:31:17.236898 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-e6975ce4-bacb-4ef7-a88e-c10db58488f8 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:31:17.237283 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-e6975ce4-bacb-4ef7-a88e-c10db58488f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:31:17.252629 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 754/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 7 15:31:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:31:17.298252 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb435f-1808-4997-bc68-1882162ac93e None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:31:17.300753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-2bcb435f-1808-4997-bc68-1882162ac93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:31:17.326316 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb435f-1808-4997-bc68-1882162ac93e None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:31:17.329990 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d84bd45-9858-4563-946b-5ec080432711 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:31:17.330635 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84bd45-9858-4563-946b-5ec080432711 service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:31:17.330919 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84bd45-9858-4563-946b-5ec080432711 service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:31:17.331194 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84bd45-9858-4563-946b-5ec080432711 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:31:17.349931 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d84bd45-9858-4563-946b-5ec080432711 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:31:17.351371 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d84bd45-9858-4563-946b-5ec080432711 service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:31:17.361445 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 779/1533] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:31:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:31:20.882183 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53683c61-6767-4291-b2bf-481560d4cb9b None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:31:20.882685 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53683c61-6767-4291-b2bf-481560d4cb9b None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:31:20.882685 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53683c61-6767-4291-b2bf-481560d4cb9b None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:31:20.884053 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 755/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 7 15:31:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:31:20.889334 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bdc209-5cdc-49b4-ba46-679b237ffe11 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:31:20.889792 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bdc209-5cdc-49b4-ba46-679b237ffe11 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:31:20.889935 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bdc209-5cdc-49b4-ba46-679b237ffe11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:31:20.909206 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-49bdc209-5cdc-49b4-ba46-679b237ffe11 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:31:20.910666 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-49bdc209-5cdc-49b4-ba46-679b237ffe11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:31:20.918981 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 780/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 7 15:31:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:31:21.220999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53683c61-6767-4291-b2bf-481560d4cb9b None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:31:21.222944 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-53683c61-6767-4291-b2bf-481560d4cb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:31:21.285572 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53683c61-6767-4291-b2bf-481560d4cb9b None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:31:21.289081 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbb064d-828b-45e2-bde5-c4d4262a5d54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:31:21.289728 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb064d-828b-45e2-bde5-c4d4262a5d54 service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:31:21.290012 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb064d-828b-45e2-bde5-c4d4262a5d54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:31:21.290289 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb064d-828b-45e2-bde5-c4d4262a5d54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:31:21.299189 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbb064d-828b-45e2-bde5-c4d4262a5d54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:31:21.301179 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbb064d-828b-45e2-bde5-c4d4262a5d54 service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:31:21.310594 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 756/1536] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:31:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:31:23.354921 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3b4d6-e3e6-4051-93a9-93ba8717c8a2 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:31:23.355520 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3b4d6-e3e6-4051-93a9-93ba8717c8a2 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:31:23.355520 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3b4d6-e3e6-4051-93a9-93ba8717c8a2 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:31:23.357136 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 781/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 7 15:31:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:31:23.362233 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a4a13-0875-4873-92e5-e3375277b9c9 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:31:23.367170 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a4a13-0875-4873-92e5-e3375277b9c9 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:31:23.367170 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a4a13-0875-4873-92e5-e3375277b9c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:31:23.384881 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-501a4a13-0875-4873-92e5-e3375277b9c9 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:31:23.386092 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-501a4a13-0875-4873-92e5-e3375277b9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:31:23.393955 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 757/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 7 15:31:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:31:23.694651 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a3b4d6-e3e6-4051-93a9-93ba8717c8a2 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:31:23.696352 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-e2a3b4d6-e3e6-4051-93a9-93ba8717c8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:31:23.704041 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=65 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:31:23.710026 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a3b4d6-e3e6-4051-93a9-93ba8717c8a2 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:31:23.716448 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff367ddd-e1c9-40a1-9287-d9dcc49ceb3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:31:23.716448 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff367ddd-e1c9-40a1-9287-d9dcc49ceb3c service cinder] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:31:23.716448 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff367ddd-e1c9-40a1-9287-d9dcc49ceb3c service cinder] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:31:23.716448 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff367ddd-e1c9-40a1-9287-d9dcc49ceb3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:31:23.724886 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff367ddd-e1c9-40a1-9287-d9dcc49ceb3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:31:23.726313 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff367ddd-e1c9-40a1-9287-d9dcc49ceb3c service cinder] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:31:23.736246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 782/1539] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:31:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:31:33.380004 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=32 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:31:33.737377 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=5 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:32:54.719761 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44a74b-5095-462f-ae8a-4be3368ea9c4 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:54.720847 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44a74b-5095-462f-ae8a-4be3368ea9c4 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:54.721367 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44a74b-5095-462f-ae8a-4be3368ea9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:54.753139 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-dd44a74b-5095-462f-ae8a-4be3368ea9c4 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:32:54.754882 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-dd44a74b-5095-462f-ae8a-4be3368ea9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:32:54.758604 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd4344-a4d3-43e1-b1d9-ce65297c717c None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:54.759676 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd4344-a4d3-43e1-b1d9-ce65297c717c None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:54.760190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd4344-a4d3-43e1-b1d9-ce65297c717c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:54.765940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 758/1540] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:32:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:32:54.776983 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c2257-25ca-4e81-930c-0800eb8caf67 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:54.777297 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c2257-25ca-4e81-930c-0800eb8caf67 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:54.777546 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c2257-25ca-4e81-930c-0800eb8caf67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:54.786958 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-edfd4344-a4d3-43e1-b1d9-ce65297c717c None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:32:54.788564 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-edfd4344-a4d3-43e1-b1d9-ce65297c717c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:32:54.807944 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 783/1541] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:32:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:32:54.809572 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ba3fe-7c64-403a-85bc-7ceb8df3565d None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:54.809699 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-642c2257-25ca-4e81-930c-0800eb8caf67 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:32:54.810246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ba3fe-7c64-403a-85bc-7ceb8df3565d None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:54.810752 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ba3fe-7c64-403a-85bc-7ceb8df3565d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:54.810882 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-642c2257-25ca-4e81-930c-0800eb8caf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:32:54.831677 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 759/1542] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:32:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:32:54.837461 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c2257-25ca-4e81-930c-0800eb8caf67 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:54.838695 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-942ba3fe-7c64-403a-85bc-7ceb8df3565d None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:32:54.840172 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-942ba3fe-7c64-403a-85bc-7ceb8df3565d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:32:54.846964 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c2257-25ca-4e81-930c-0800eb8caf67 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:54.853565 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 784/1543] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:32:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:32:54.855188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e653275b-77a2-4d4b-becd-5496008abb84 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:54.855740 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e653275b-77a2-4d4b-becd-5496008abb84 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:54.856497 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e653275b-77a2-4d4b-becd-5496008abb84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:54.872430 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ed368-2b65-4f57-842e-3992371c0af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:54.872430 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ed368-2b65-4f57-842e-3992371c0af7 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:54.872739 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ed368-2b65-4f57-842e-3992371c0af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:54.872979 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ed368-2b65-4f57-842e-3992371c0af7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:54.873641 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ed368-2b65-4f57-842e-3992371c0af7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:32:54.873953 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ed368-2b65-4f57-842e-3992371c0af7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:54.879146 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ed368-2b65-4f57-842e-3992371c0af7 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:54.889344 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-e653275b-77a2-4d4b-becd-5496008abb84 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:32:54.890916 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-e653275b-77a2-4d4b-becd-5496008abb84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:32:54.901835 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 785/1545] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:32:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:32:54.902134 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 760/1545] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:32:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:32:54.903656 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa935703-1ad4-44c3-a1b9-25c67fc6dda4 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:54.905898 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa935703-1ad4-44c3-a1b9-25c67fc6dda4 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:54.905898 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa935703-1ad4-44c3-a1b9-25c67fc6dda4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:54.907444 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e653275b-77a2-4d4b-becd-5496008abb84 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:54.917287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e653275b-77a2-4d4b-becd-5496008abb84 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:54.922343 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384af026-41d7-4962-8a6e-efc5de19d6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:54.923913 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384af026-41d7-4962-8a6e-efc5de19d6f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:54.924451 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384af026-41d7-4962-8a6e-efc5de19d6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:54.924946 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384af026-41d7-4962-8a6e-efc5de19d6f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:54.926053 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384af026-41d7-4962-8a6e-efc5de19d6f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:32:54.926580 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384af026-41d7-4962-8a6e-efc5de19d6f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:54.928741 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: WARNING py.warnings [None req-384af026-41d7-4962-8a6e-efc5de19d6f8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 07 15:32:54.928741 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: warnings.warn(msg) Jul 07 15:32:54.928741 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: Jul 07 15:32:54.930209 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384af026-41d7-4962-8a6e-efc5de19d6f8 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:54.940604 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-fa935703-1ad4-44c3-a1b9-25c67fc6dda4 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:32:54.942213 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-fa935703-1ad4-44c3-a1b9-25c67fc6dda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:32:54.949640 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 786/1546] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:32:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:32:54.949640 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384af026-41d7-4962-8a6e-efc5de19d6f8 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:54.964482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 761/1547] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:32:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:32:54.965846 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c4db3-102a-41ce-9270-43cc5802b352 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:54.966138 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c4db3-102a-41ce-9270-43cc5802b352 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:54.966421 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c4db3-102a-41ce-9270-43cc5802b352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:54.975708 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384af026-41d7-4962-8a6e-efc5de19d6f8 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:54.984963 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7051ea92-d87f-400f-90ec-85c84a03967b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:54.986620 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7051ea92-d87f-400f-90ec-85c84a03967b admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:54.987594 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7051ea92-d87f-400f-90ec-85c84a03967b admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:54.990235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7051ea92-d87f-400f-90ec-85c84a03967b admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:54.995540 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7051ea92-d87f-400f-90ec-85c84a03967b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:54.997871 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7051ea92-d87f-400f-90ec-85c84a03967b admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:55.011152 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-a24c4db3-102a-41ce-9270-43cc5802b352 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:32:55.013118 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-a24c4db3-102a-41ce-9270-43cc5802b352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:32:55.026520 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 762/1548] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:32:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:32:55.028147 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350ad5f-143a-44f6-a17a-be64542be95c None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:55.028633 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350ad5f-143a-44f6-a17a-be64542be95c None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:55.029070 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350ad5f-143a-44f6-a17a-be64542be95c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:55.047248 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-a350ad5f-143a-44f6-a17a-be64542be95c None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:32:55.049016 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-a350ad5f-143a-44f6-a17a-be64542be95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:32:55.056838 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 763/1549] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:32:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:32:55.057584 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350ad5f-143a-44f6-a17a-be64542be95c None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:55.067715 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350ad5f-143a-44f6-a17a-be64542be95c None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:55.072311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea144c6-bb26-4c00-b594-68428fb93849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:55.072949 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea144c6-bb26-4c00-b594-68428fb93849 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:55.073229 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea144c6-bb26-4c00-b594-68428fb93849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:55.073501 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea144c6-bb26-4c00-b594-68428fb93849 admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:55.074269 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea144c6-bb26-4c00-b594-68428fb93849 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:55.075278 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: WARNING py.warnings [None req-9ea144c6-bb26-4c00-b594-68428fb93849 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 07 15:32:55.075278 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: warnings.warn(msg) Jul 07 15:32:55.075278 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: Jul 07 15:32:55.076492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea144c6-bb26-4c00-b594-68428fb93849 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:55.110355 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 787/1550] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:32:54 2022] POST /identity/v3/projects => generated 379 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:32:55.111830 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7051ea92-d87f-400f-90ec-85c84a03967b admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:55.172376 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7051ea92-d87f-400f-90ec-85c84a03967b admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:55.177921 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbadf12f-df60-436f-8587-1db788b78dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:55.178629 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbadf12f-df60-436f-8587-1db788b78dbc admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:55.178915 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbadf12f-df60-436f-8587-1db788b78dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:55.179190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbadf12f-df60-436f-8587-1db788b78dbc admin admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:55.179954 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbadf12f-df60-436f-8587-1db788b78dbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:32:55.180287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbadf12f-df60-436f-8587-1db788b78dbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:55.181941 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbadf12f-df60-436f-8587-1db788b78dbc admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:55.200827 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 788/1551] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:32:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:32:55.200827 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a46aec-fe5f-4c2e-8ea8-13e033a61d11 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:55.200827 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a46aec-fe5f-4c2e-8ea8-13e033a61d11 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:55.200827 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a46aec-fe5f-4c2e-8ea8-13e033a61d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:55.205919 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 764/1552] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:32:55 2022] POST /identity/v3/projects => generated 385 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:32:55.205919 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219ec26-93d6-46cb-8c1e-052daf5b8c9b None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:55.205919 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219ec26-93d6-46cb-8c1e-052daf5b8c9b None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:55.205919 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219ec26-93d6-46cb-8c1e-052daf5b8c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:55.219195 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-a4a46aec-fe5f-4c2e-8ea8-13e033a61d11 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:32:55.220968 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-a4a46aec-fe5f-4c2e-8ea8-13e033a61d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:32:55.224472 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-a219ec26-93d6-46cb-8c1e-052daf5b8c9b None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:32:55.226904 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-a219ec26-93d6-46cb-8c1e-052daf5b8c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:32:55.280210 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 789/1553] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:32:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:32:55.281596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1ffdd-59f1-47aa-947c-a66889c29b2e None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:55.281895 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1ffdd-59f1-47aa-947c-a66889c29b2e None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:55.282171 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1ffdd-59f1-47aa-947c-a66889c29b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:55.285897 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 765/1554] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:32:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:32:55.287003 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b9cea-65df-46a4-a0d0-2c00ef39c911 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:55.287293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b9cea-65df-46a4-a0d0-2c00ef39c911 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:55.287737 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b9cea-65df-46a4-a0d0-2c00ef39c911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:55.307939 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-895b9cea-65df-46a4-a0d0-2c00ef39c911 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:32:55.309179 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-895b9cea-65df-46a4-a0d0-2c00ef39c911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:32:55.314909 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-10f1ffdd-59f1-47aa-947c-a66889c29b2e None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:32:55.314909 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-10f1ffdd-59f1-47aa-947c-a66889c29b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:32:55.316970 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 766/1555] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:32:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:32:55.318177 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c72b8cf-6732-485a-9feb-c11d18c2cc1c None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:55.318461 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c72b8cf-6732-485a-9feb-c11d18c2cc1c None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:55.318735 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c72b8cf-6732-485a-9feb-c11d18c2cc1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:55.321356 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 790/1556] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:32:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:32:55.322165 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f1ffdd-59f1-47aa-947c-a66889c29b2e None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:55.331499 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f1ffdd-59f1-47aa-947c-a66889c29b2e None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:55.336097 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c53987-0986-45c7-9589-ee594d419272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:55.336753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c53987-0986-45c7-9589-ee594d419272 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:55.336889 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-1c72b8cf-6732-485a-9feb-c11d18c2cc1c None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:32:55.337040 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c53987-0986-45c7-9589-ee594d419272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:55.337317 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c53987-0986-45c7-9589-ee594d419272 admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:55.338030 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c53987-0986-45c7-9589-ee594d419272 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:55.338143 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-1c72b8cf-6732-485a-9feb-c11d18c2cc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:32:55.339050 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: WARNING py.warnings [None req-04c53987-0986-45c7-9589-ee594d419272 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 07 15:32:55.339050 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: warnings.warn(msg) Jul 07 15:32:55.339050 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: Jul 07 15:32:55.340289 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c53987-0986-45c7-9589-ee594d419272 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:55.346621 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 767/1557] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:32:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:32:55.351781 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c72b8cf-6732-485a-9feb-c11d18c2cc1c None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:55.358286 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c72b8cf-6732-485a-9feb-c11d18c2cc1c None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:55.366607 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099e108-6e42-4a2f-87d1-d8effdcbc0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:55.366607 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099e108-6e42-4a2f-87d1-d8effdcbc0ba admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:55.366607 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099e108-6e42-4a2f-87d1-d8effdcbc0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:55.366607 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099e108-6e42-4a2f-87d1-d8effdcbc0ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:55.367092 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4099e108-6e42-4a2f-87d1-d8effdcbc0ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:55.367092 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4099e108-6e42-4a2f-87d1-d8effdcbc0ba admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:55.448920 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 791/1558] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:32:55 2022] POST /identity/v3/users => generated 290 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:32:55.449898 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c53987-0986-45c7-9589-ee594d419272 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:55.459806 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c53987-0986-45c7-9589-ee594d419272 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:55.464519 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f61e67-c04a-4a5a-a912-b519d129e8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:55.465081 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f61e67-c04a-4a5a-a912-b519d129e8d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:55.465370 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f61e67-c04a-4a5a-a912-b519d129e8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:55.465673 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f61e67-c04a-4a5a-a912-b519d129e8d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:55.466605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f61e67-c04a-4a5a-a912-b519d129e8d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:55.468274 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f61e67-c04a-4a5a-a912-b519d129e8d5 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:55.474373 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 768/1559] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:32:55 2022] POST /identity/v3/projects => generated 367 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:32:55.476879 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098dbb2-34a9-40df-8521-754ba91125b9 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:55.477247 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098dbb2-34a9-40df-8521-754ba91125b9 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:55.477516 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098dbb2-34a9-40df-8521-754ba91125b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:55.495148 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-e098dbb2-34a9-40df-8521-754ba91125b9 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:32:55.496417 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-e098dbb2-34a9-40df-8521-754ba91125b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:32:55.545243 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:32:55.554442 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 769/1560] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:32:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:32:55.555453 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:32:55.556788 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74655454-36a2-4008-8147-9291e91575f2 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:55.556788 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74655454-36a2-4008-8147-9291e91575f2 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:55.557078 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74655454-36a2-4008-8147-9291e91575f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:55.561722 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 792/1561] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:32:55 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:32:55.563692 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f61e67-c04a-4a5a-a912-b519d129e8d5 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:55.571913 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f61e67-c04a-4a5a-a912-b519d129e8d5 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:55.576011 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-74655454-36a2-4008-8147-9291e91575f2 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:32:55.576447 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb4b979-6281-4e2f-b3b2-50059367f4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:55.577091 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4b979-6281-4e2f-b3b2-50059367f4aa admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:55.577216 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-74655454-36a2-4008-8147-9291e91575f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:32:55.577372 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4b979-6281-4e2f-b3b2-50059367f4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:55.577650 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4b979-6281-4e2f-b3b2-50059367f4aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:55.578368 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb4b979-6281-4e2f-b3b2-50059367f4aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:32:55.578705 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb4b979-6281-4e2f-b3b2-50059367f4aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:55.581173 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb4b979-6281-4e2f-b3b2-50059367f4aa admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:55.589326 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 770/1562] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:32:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:32:55.589326 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74655454-36a2-4008-8147-9291e91575f2 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:55.596887 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 793/1563] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:32:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:32:55.596887 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea983fa-1928-4f7f-af9a-6ebf8ee1118a None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:55.596887 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea983fa-1928-4f7f-af9a-6ebf8ee1118a None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:55.596887 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea983fa-1928-4f7f-af9a-6ebf8ee1118a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:55.597545 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74655454-36a2-4008-8147-9291e91575f2 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:55.602342 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf05f2-f586-4354-ba3b-200878db1156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:55.602988 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf05f2-f586-4354-ba3b-200878db1156 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:55.603489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf05f2-f586-4354-ba3b-200878db1156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:55.603644 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf05f2-f586-4354-ba3b-200878db1156 admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:55.604303 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bf05f2-f586-4354-ba3b-200878db1156 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:32:55.604606 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bf05f2-f586-4354-ba3b-200878db1156 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:55.606448 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bf05f2-f586-4354-ba3b-200878db1156 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:55.618952 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 771/1564] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:32:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:32:55.619845 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf05f2-f586-4354-ba3b-200878db1156 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:55.623989 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-1ea983fa-1928-4f7f-af9a-6ebf8ee1118a None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:32:55.625254 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-1ea983fa-1928-4f7f-af9a-6ebf8ee1118a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:32:55.629285 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf05f2-f586-4354-ba3b-200878db1156 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:55.633640 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 794/1565] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:32:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:32:55.635053 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8bbc0-3a7c-4dd3-8986-472fe87905cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:55.641716 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea983fa-1928-4f7f-af9a-6ebf8ee1118a None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:55.641855 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8bbc0-3a7c-4dd3-8986-472fe87905cb admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:55.641855 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8bbc0-3a7c-4dd3-8986-472fe87905cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:55.641855 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8bbc0-3a7c-4dd3-8986-472fe87905cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:55.641855 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c8bbc0-3a7c-4dd3-8986-472fe87905cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:55.641855 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c8bbc0-3a7c-4dd3-8986-472fe87905cb admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:55.645035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea983fa-1928-4f7f-af9a-6ebf8ee1118a None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:55.651621 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 772/1566] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:32:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:32:55.651621 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8bbc0-3a7c-4dd3-8986-472fe87905cb admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:55.653806 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b482812-e481-4e46-bcf3-b4423184b5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:55.654510 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b482812-e481-4e46-bcf3-b4423184b5d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:55.654795 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b482812-e481-4e46-bcf3-b4423184b5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:55.655100 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b482812-e481-4e46-bcf3-b4423184b5d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:55.655822 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b482812-e481-4e46-bcf3-b4423184b5d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:55.658603 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b482812-e481-4e46-bcf3-b4423184b5d0 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:55.661782 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8bbc0-3a7c-4dd3-8986-472fe87905cb admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:55.666554 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914ab11-28c1-4253-8e05-9f58f3567abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:55.667195 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914ab11-28c1-4253-8e05-9f58f3567abb admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:55.667472 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914ab11-28c1-4253-8e05-9f58f3567abb admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:55.667776 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914ab11-28c1-4253-8e05-9f58f3567abb admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:55.668525 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3914ab11-28c1-4253-8e05-9f58f3567abb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:32:55.668833 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3914ab11-28c1-4253-8e05-9f58f3567abb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:55.670383 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3914ab11-28c1-4253-8e05-9f58f3567abb admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:55.681907 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 773/1567] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:32:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:32:55.682681 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914ab11-28c1-4253-8e05-9f58f3567abb admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:55.691722 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914ab11-28c1-4253-8e05-9f58f3567abb admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:55.696778 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d131fc8f-c0fe-4c12-b3a1-4e42b7b1d6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:55.697470 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131fc8f-c0fe-4c12-b3a1-4e42b7b1d6e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:55.697750 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131fc8f-c0fe-4c12-b3a1-4e42b7b1d6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:55.698021 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131fc8f-c0fe-4c12-b3a1-4e42b7b1d6e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:55.698719 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d131fc8f-c0fe-4c12-b3a1-4e42b7b1d6e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:55.700279 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d131fc8f-c0fe-4c12-b3a1-4e42b7b1d6e0 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:55.710613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 774/1568] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:32:55 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:32:55.711385 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d131fc8f-c0fe-4c12-b3a1-4e42b7b1d6e0 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:55.723853 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d131fc8f-c0fe-4c12-b3a1-4e42b7b1d6e0 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:55.728589 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b3df2-0cc1-4f7a-b238-c02b954fdb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:55.729173 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b3df2-0cc1-4f7a-b238-c02b954fdb63 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:55.729450 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b3df2-0cc1-4f7a-b238-c02b954fdb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:55.729719 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b3df2-0cc1-4f7a-b238-c02b954fdb63 admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:55.730430 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b3df2-0cc1-4f7a-b238-c02b954fdb63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:32:55.730732 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b3df2-0cc1-4f7a-b238-c02b954fdb63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:55.732670 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b3df2-0cc1-4f7a-b238-c02b954fdb63 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:55.745083 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 775/1569] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:32:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:32:55.745083 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b3df2-0cc1-4f7a-b238-c02b954fdb63 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:55.754542 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 795/1570] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:32:55 2022] POST /identity/v3/users => generated 284 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:32:55.754542 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b482812-e481-4e46-bcf3-b4423184b5d0 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:55.757696 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b3df2-0cc1-4f7a-b238-c02b954fdb63 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:55.759840 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b482812-e481-4e46-bcf3-b4423184b5d0 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:55.762578 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b31820d-67c3-4855-b805-6ac9b388d5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:55.763222 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31820d-67c3-4855-b805-6ac9b388d5c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:55.763504 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31820d-67c3-4855-b805-6ac9b388d5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:55.763817 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31820d-67c3-4855-b805-6ac9b388d5c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:55.764372 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba8802-2e21-49c1-8d6e-34264f1577d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:55.764526 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b31820d-67c3-4855-b805-6ac9b388d5c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:55.765012 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba8802-2e21-49c1-8d6e-34264f1577d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:55.765292 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba8802-2e21-49c1-8d6e-34264f1577d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:55.765567 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba8802-2e21-49c1-8d6e-34264f1577d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:55.766254 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba8802-2e21-49c1-8d6e-34264f1577d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:55.766577 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b31820d-67c3-4855-b805-6ac9b388d5c7 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:55.768672 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba8802-2e21-49c1-8d6e-34264f1577d4 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:55.881990 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 776/1571] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:32:55 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:32:55.883578 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b31820d-67c3-4855-b805-6ac9b388d5c7 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:55.910812 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 796/1572] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:32:55 2022] POST /identity/v3/projects => generated 375 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:32:55.911679 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba8802-2e21-49c1-8d6e-34264f1577d4 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:55.948052 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b31820d-67c3-4855-b805-6ac9b388d5c7 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:55.977592 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba8802-2e21-49c1-8d6e-34264f1577d4 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:55.982782 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63864a22-32fa-48be-8eb2-4eaf55976803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:55.983388 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63864a22-32fa-48be-8eb2-4eaf55976803 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:55.983701 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63864a22-32fa-48be-8eb2-4eaf55976803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:55.983977 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63864a22-32fa-48be-8eb2-4eaf55976803 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:55.984714 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63864a22-32fa-48be-8eb2-4eaf55976803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:55.986330 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63864a22-32fa-48be-8eb2-4eaf55976803 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:55.996094 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 797/1573] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:32:55 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:32:55.996851 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63864a22-32fa-48be-8eb2-4eaf55976803 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:56.006600 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63864a22-32fa-48be-8eb2-4eaf55976803 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:56.010248 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609c498-6c15-46d2-b18a-012e2e84cb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:56.010905 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609c498-6c15-46d2-b18a-012e2e84cb0a admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:56.011187 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609c498-6c15-46d2-b18a-012e2e84cb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:56.011286 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f8baa-18c4-42f9-a3fb-96c88516608b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:56.011461 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609c498-6c15-46d2-b18a-012e2e84cb0a admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:56.011908 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f8baa-18c4-42f9-a3fb-96c88516608b admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:56.012188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f8baa-18c4-42f9-a3fb-96c88516608b admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:56.012296 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f609c498-6c15-46d2-b18a-012e2e84cb0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:32:56.012479 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f8baa-18c4-42f9-a3fb-96c88516608b admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:56.012589 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f609c498-6c15-46d2-b18a-012e2e84cb0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:56.013128 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6f8baa-18c4-42f9-a3fb-96c88516608b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:56.014067 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f609c498-6c15-46d2-b18a-012e2e84cb0a admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:56.015008 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6f8baa-18c4-42f9-a3fb-96c88516608b admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:56.025915 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 777/1574] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:32:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:32:56.026703 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609c498-6c15-46d2-b18a-012e2e84cb0a admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:56.036249 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609c498-6c15-46d2-b18a-012e2e84cb0a admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:56.040727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed3ce9-c986-48b0-acb1-9de2fbfea914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:56.041362 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed3ce9-c986-48b0-acb1-9de2fbfea914 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:56.041646 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed3ce9-c986-48b0-acb1-9de2fbfea914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:56.041919 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed3ce9-c986-48b0-acb1-9de2fbfea914 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:56.042626 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ed3ce9-c986-48b0-acb1-9de2fbfea914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:56.044213 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ed3ce9-c986-48b0-acb1-9de2fbfea914 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:56.046210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:32:56.055458 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 778/1575] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:32:56.056307 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed3ce9-c986-48b0-acb1-9de2fbfea914 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:56.065785 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed3ce9-c986-48b0-acb1-9de2fbfea914 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:56.070211 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e92d276-b7e1-4c02-a70a-85aff4fc90dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:56.070807 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e92d276-b7e1-4c02-a70a-85aff4fc90dd admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:56.071088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e92d276-b7e1-4c02-a70a-85aff4fc90dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:56.071360 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e92d276-b7e1-4c02-a70a-85aff4fc90dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:56.072073 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e92d276-b7e1-4c02-a70a-85aff4fc90dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:56.073900 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e92d276-b7e1-4c02-a70a-85aff4fc90dd admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:56.093091 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 798/1576] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:32:55 2022] POST /identity/v3/projects => generated 363 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:32:56.093917 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f8baa-18c4-42f9-a3fb-96c88516608b admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:56.155430 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f8baa-18c4-42f9-a3fb-96c88516608b admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:56.160134 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8ccc1-f651-4b02-aa6a-a17f0e52882a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:56.160820 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8ccc1-f651-4b02-aa6a-a17f0e52882a admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:56.161223 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8ccc1-f651-4b02-aa6a-a17f0e52882a admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:56.161511 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8ccc1-f651-4b02-aa6a-a17f0e52882a admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:56.162242 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f8ccc1-f651-4b02-aa6a-a17f0e52882a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:56.163877 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f8ccc1-f651-4b02-aa6a-a17f0e52882a admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:56.173667 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 799/1577] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:32:56.174420 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8ccc1-f651-4b02-aa6a-a17f0e52882a admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:56.183872 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8ccc1-f651-4b02-aa6a-a17f0e52882a admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:56.188373 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 779/1578] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:32:56 2022] POST /identity/v3/projects => generated 383 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:32:56.188373 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e92d276-b7e1-4c02-a70a-85aff4fc90dd admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:56.188627 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:32:56.240452 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b21d80d-755d-4d95-9a81-0c0f0c96a5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:56.241044 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21d80d-755d-4d95-9a81-0c0f0c96a5d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:56.241323 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21d80d-755d-4d95-9a81-0c0f0c96a5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:56.241597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21d80d-755d-4d95-9a81-0c0f0c96a5d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:56.242272 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b21d80d-755d-4d95-9a81-0c0f0c96a5d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:56.244227 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b21d80d-755d-4d95-9a81-0c0f0c96a5d9 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:56.252031 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e92d276-b7e1-4c02-a70a-85aff4fc90dd admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:56.256542 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3becf01f-2afe-4722-9ee3-601efefb0ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:56.257199 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3becf01f-2afe-4722-9ee3-601efefb0ab9 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:56.257480 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3becf01f-2afe-4722-9ee3-601efefb0ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:56.257751 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3becf01f-2afe-4722-9ee3-601efefb0ab9 admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:56.258426 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3becf01f-2afe-4722-9ee3-601efefb0ab9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:56.260340 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3becf01f-2afe-4722-9ee3-601efefb0ab9 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:56.352162 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 800/1579] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:32:56 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:32:56.353132 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b21d80d-755d-4d95-9a81-0c0f0c96a5d9 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:56.362806 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b21d80d-755d-4d95-9a81-0c0f0c96a5d9 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:56.367306 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056203d8-1094-493e-aee1-ced694e94286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:56.368186 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056203d8-1094-493e-aee1-ced694e94286 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:56.368470 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056203d8-1094-493e-aee1-ced694e94286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:56.368888 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056203d8-1094-493e-aee1-ced694e94286 admin admin] PATH_INFO: `/v3/projects/b78a55fc7dba4ed5b22675b585cfe3aa/users/332b1c9c68db4175a1383bb588b5e822/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:56.374621 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 780/1580] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:32:56 2022] POST /identity/v3/users => generated 288 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:32:56.375287 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3becf01f-2afe-4722-9ee3-601efefb0ab9 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:56.384533 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056203d8-1094-493e-aee1-ced694e94286 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b78a55fc7dba4ed5b22675b585cfe3aa, user_id=332b1c9c68db4175a1383bb588b5e822, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:56.384846 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3becf01f-2afe-4722-9ee3-601efefb0ab9 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:56.387191 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056203d8-1094-493e-aee1-ced694e94286 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:56.389286 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2e138-874c-470e-85d2-14ca1847da5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:56.390027 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2e138-874c-470e-85d2-14ca1847da5a admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:56.390231 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2e138-874c-470e-85d2-14ca1847da5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:56.390500 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2e138-874c-470e-85d2-14ca1847da5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:56.391169 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b2e138-874c-470e-85d2-14ca1847da5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:56.393564 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b2e138-874c-470e-85d2-14ca1847da5a admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:56.492447 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 801/1581] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:32:56 2022] PUT /identity/v3/projects/b78a55fc7dba4ed5b22675b585cfe3aa/users/332b1c9c68db4175a1383bb588b5e822/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:32:56.493162 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056203d8-1094-493e-aee1-ced694e94286 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:56.524874 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 781/1582] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:32:56 2022] POST /identity/v3/projects => generated 405 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:32:56.524874 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2e138-874c-470e-85d2-14ca1847da5a admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:56.557005 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056203d8-1094-493e-aee1-ced694e94286 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:56.583042 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2e138-874c-470e-85d2-14ca1847da5a admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:56.588128 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d656423-7df0-4cf6-ae73-9c74d408e0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:56.588788 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d656423-7df0-4cf6-ae73-9c74d408e0b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:56.589067 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d656423-7df0-4cf6-ae73-9c74d408e0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:56.589453 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d656423-7df0-4cf6-ae73-9c74d408e0b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:56.590138 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d656423-7df0-4cf6-ae73-9c74d408e0b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:56.592067 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d656423-7df0-4cf6-ae73-9c74d408e0b8 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:56.613296 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9caaf8-f825-41ba-909c-42a4544a86b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:56.614186 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9caaf8-f825-41ba-909c-42a4544a86b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:56.614469 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9caaf8-f825-41ba-909c-42a4544a86b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:56.614746 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9caaf8-f825-41ba-909c-42a4544a86b5 admin admin] PATH_INFO: `/v3/projects/d2e4afda121042939745304ce6f76a8b/users/2033ad40d45747aa8294e4feb5a6cfe0/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:56.629934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9caaf8-f825-41ba-909c-42a4544a86b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2e4afda121042939745304ce6f76a8b, user_id=2033ad40d45747aa8294e4feb5a6cfe0, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:56.632612 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9caaf8-f825-41ba-909c-42a4544a86b5 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:56.697772 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 782/1583] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:32:56 2022] POST /identity/v3/users => generated 282 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:32:56.698660 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d656423-7df0-4cf6-ae73-9c74d408e0b8 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:56.708346 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d656423-7df0-4cf6-ae73-9c74d408e0b8 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:56.712755 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400fa27-6601-46d1-a534-a7ac1ba024cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:56.713419 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400fa27-6601-46d1-a534-a7ac1ba024cd admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:56.713696 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400fa27-6601-46d1-a534-a7ac1ba024cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:56.713970 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400fa27-6601-46d1-a534-a7ac1ba024cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:56.714790 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0400fa27-6601-46d1-a534-a7ac1ba024cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:56.716393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0400fa27-6601-46d1-a534-a7ac1ba024cd admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:56.726983 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 783/1584] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:32:56.733190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 802/1585] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:32:56 2022] PUT /identity/v3/projects/d2e4afda121042939745304ce6f76a8b/users/2033ad40d45747aa8294e4feb5a6cfe0/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:32:56.733190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9caaf8-f825-41ba-909c-42a4544a86b5 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:56.733418 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400fa27-6601-46d1-a534-a7ac1ba024cd admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:56.735186 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:32:56.740026 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400fa27-6601-46d1-a534-a7ac1ba024cd admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:56.758998 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:32:56.791771 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9caaf8-f825-41ba-909c-42a4544a86b5 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:56.796540 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ada1f-4fa2-45fc-bb06-86564264a435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:56.796948 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb25508-86f5-42f3-8d87-15e40e7db7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:56.797219 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ada1f-4fa2-45fc-bb06-86564264a435 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:56.797494 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ada1f-4fa2-45fc-bb06-86564264a435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:56.797602 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb25508-86f5-42f3-8d87-15e40e7db7ea admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:56.797763 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ada1f-4fa2-45fc-bb06-86564264a435 admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:56.797864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb25508-86f5-42f3-8d87-15e40e7db7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:56.798123 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb25508-86f5-42f3-8d87-15e40e7db7ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:56.799119 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ada1f-4fa2-45fc-bb06-86564264a435 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:56.799483 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb25508-86f5-42f3-8d87-15e40e7db7ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:56.801143 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb25508-86f5-42f3-8d87-15e40e7db7ea admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:56.801255 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ada1f-4fa2-45fc-bb06-86564264a435 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:56.811221 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 784/1586] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:32:56.812154 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb25508-86f5-42f3-8d87-15e40e7db7ea admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:56.822039 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb25508-86f5-42f3-8d87-15e40e7db7ea admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:56.826693 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10efb06-d046-4043-96ef-df4d3e71729e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:56.827436 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10efb06-d046-4043-96ef-df4d3e71729e admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:56.827771 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10efb06-d046-4043-96ef-df4d3e71729e admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:56.828045 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10efb06-d046-4043-96ef-df4d3e71729e admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:56.828745 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10efb06-d046-4043-96ef-df4d3e71729e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:56.830487 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10efb06-d046-4043-96ef-df4d3e71729e admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:56.840096 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 785/1587] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:32:56.841406 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265b818-f8d1-48d4-82b1-74390576c15a None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:56.845759 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265b818-f8d1-48d4-82b1-74390576c15a None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:56.845759 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265b818-f8d1-48d4-82b1-74390576c15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:56.859884 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-7265b818-f8d1-48d4-82b1-74390576c15a None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:32:56.861760 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-7265b818-f8d1-48d4-82b1-74390576c15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:32:56.866346 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 803/1588] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:32:56 2022] POST /identity/v3/users => generated 292 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:32:56.868278 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ada1f-4fa2-45fc-bb06-86564264a435 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:56.869661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 786/1589] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:32:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:32:56.870643 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265b818-f8d1-48d4-82b1-74390576c15a None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:56.880300 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265b818-f8d1-48d4-82b1-74390576c15a None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:56.880492 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ada1f-4fa2-45fc-bb06-86564264a435 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:56.884888 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873df08-fa81-4aca-8497-a121c9171ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:56.885047 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5200bee-6fb7-4d4c-8531-4daee73d0c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:56.885567 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873df08-fa81-4aca-8497-a121c9171ac4 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:56.885688 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5200bee-6fb7-4d4c-8531-4daee73d0c2b admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:56.885854 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873df08-fa81-4aca-8497-a121c9171ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:56.885986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5200bee-6fb7-4d4c-8531-4daee73d0c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:56.886178 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873df08-fa81-4aca-8497-a121c9171ac4 admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:56.886292 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5200bee-6fb7-4d4c-8531-4daee73d0c2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:56.886909 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5200bee-6fb7-4d4c-8531-4daee73d0c2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:56.887017 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0873df08-fa81-4aca-8497-a121c9171ac4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:56.888964 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0873df08-fa81-4aca-8497-a121c9171ac4 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:56.891810 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5200bee-6fb7-4d4c-8531-4daee73d0c2b admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:56.899687 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 787/1590] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:32:56.901052 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae259a-ca15-49a0-9c9d-25e54e7097b3 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:56.901344 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae259a-ca15-49a0-9c9d-25e54e7097b3 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:56.901615 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae259a-ca15-49a0-9c9d-25e54e7097b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:56.919200 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-8dae259a-ca15-49a0-9c9d-25e54e7097b3 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:32:56.920394 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-8dae259a-ca15-49a0-9c9d-25e54e7097b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:32:56.928738 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 788/1591] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:32:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:32:56.929458 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae259a-ca15-49a0-9c9d-25e54e7097b3 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:56.938522 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae259a-ca15-49a0-9c9d-25e54e7097b3 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:56.943083 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919578ad-4f84-40da-88dc-97db73604fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:56.943960 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919578ad-4f84-40da-88dc-97db73604fb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:56.944257 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919578ad-4f84-40da-88dc-97db73604fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:56.944509 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919578ad-4f84-40da-88dc-97db73604fb1 admin admin] PATH_INFO: `/v3/projects/4aa164cff08d4e7f8206ee70c01e08ed/users/0a5b43ce22cc424e87598501e12d256d/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:56.959976 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919578ad-4f84-40da-88dc-97db73604fb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4aa164cff08d4e7f8206ee70c01e08ed, user_id=0a5b43ce22cc424e87598501e12d256d, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:56.961099 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: WARNING py.warnings [None req-919578ad-4f84-40da-88dc-97db73604fb1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 07 15:32:56.961099 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: warnings.warn(msg) Jul 07 15:32:56.961099 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: Jul 07 15:32:56.963136 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919578ad-4f84-40da-88dc-97db73604fb1 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:56.964063 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 804/1592] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:32:56 2022] POST /identity/v3/users => generated 303 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:32:56.967765 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873df08-fa81-4aca-8497-a121c9171ac4 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:56.978240 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873df08-fa81-4aca-8497-a121c9171ac4 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:56.982990 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0bab9d-0337-4041-8187-ba9cfbb2fec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:56.983670 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0bab9d-0337-4041-8187-ba9cfbb2fec7 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:56.983955 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0bab9d-0337-4041-8187-ba9cfbb2fec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:56.984231 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0bab9d-0337-4041-8187-ba9cfbb2fec7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:56.984954 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0bab9d-0337-4041-8187-ba9cfbb2fec7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:56.986535 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0bab9d-0337-4041-8187-ba9cfbb2fec7 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:56.996150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 805/1593] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:32:56.996891 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0bab9d-0337-4041-8187-ba9cfbb2fec7 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:57.005915 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0bab9d-0337-4041-8187-ba9cfbb2fec7 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:57.010534 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71368855-689b-4005-a9b0-635da428dbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:57.011101 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71368855-689b-4005-a9b0-635da428dbc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:57.011382 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71368855-689b-4005-a9b0-635da428dbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:57.011688 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71368855-689b-4005-a9b0-635da428dbc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:57.012394 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71368855-689b-4005-a9b0-635da428dbc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:57.014121 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71368855-689b-4005-a9b0-635da428dbc3 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:57.025128 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 806/1594] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:32:57.025942 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71368855-689b-4005-a9b0-635da428dbc3 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:57.035678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71368855-689b-4005-a9b0-635da428dbc3 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:57.041100 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 789/1595] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:32:56 2022] PUT /identity/v3/projects/4aa164cff08d4e7f8206ee70c01e08ed/users/0a5b43ce22cc424e87598501e12d256d/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:32:57.041991 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919578ad-4f84-40da-88dc-97db73604fb1 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:57.092744 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf59aa-0820-479a-9cef-68e86a2e98b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:57.093407 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf59aa-0820-479a-9cef-68e86a2e98b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:57.093690 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf59aa-0820-479a-9cef-68e86a2e98b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:57.093962 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf59aa-0820-479a-9cef-68e86a2e98b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:57.094679 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bf59aa-0820-479a-9cef-68e86a2e98b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:57.096946 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bf59aa-0820-479a-9cef-68e86a2e98b9 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:57.106824 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 807/1596] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:32:57.106984 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919578ad-4f84-40da-88dc-97db73604fb1 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:57.107702 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf59aa-0820-479a-9cef-68e86a2e98b9 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:57.110894 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69940f6-3b84-4bbf-b34f-384f114856a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:57.111536 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69940f6-3b84-4bbf-b34f-384f114856a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:57.111912 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69940f6-3b84-4bbf-b34f-384f114856a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:57.112190 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69940f6-3b84-4bbf-b34f-384f114856a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:57.116988 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf59aa-0820-479a-9cef-68e86a2e98b9 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:57.121357 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69940f6-3b84-4bbf-b34f-384f114856a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:57.121582 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b3fcc-8ce4-4e5b-b7a0-43d499e78205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:57.122207 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b3fcc-8ce4-4e5b-b7a0-43d499e78205 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:57.122485 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b3fcc-8ce4-4e5b-b7a0-43d499e78205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:57.122755 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b3fcc-8ce4-4e5b-b7a0-43d499e78205 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:57.122853 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69940f6-3b84-4bbf-b34f-384f114856a0 service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:57.123461 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566b3fcc-8ce4-4e5b-b7a0-43d499e78205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:57.125035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566b3fcc-8ce4-4e5b-b7a0-43d499e78205 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:57.132621 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 790/1597] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:32:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:32:57.133418 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69940f6-3b84-4bbf-b34f-384f114856a0 service neutron] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:57.139554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 808/1598] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:32:57.139554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b3fcc-8ce4-4e5b-b7a0-43d499e78205 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:57.141490 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69940f6-3b84-4bbf-b34f-384f114856a0 service neutron] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:57.146694 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab4b6f-5d5f-4b2d-b59c-38d1fc243e94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:57.147396 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab4b6f-5d5f-4b2d-b59c-38d1fc243e94 service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:57.147722 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b3fcc-8ce4-4e5b-b7a0-43d499e78205 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:57.147845 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab4b6f-5d5f-4b2d-b59c-38d1fc243e94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:57.148291 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab4b6f-5d5f-4b2d-b59c-38d1fc243e94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:57.152715 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6bdb4-ea89-4031-895c-682ef5b6e727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:57.153366 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6bdb4-ea89-4031-895c-682ef5b6e727 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:57.153644 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6bdb4-ea89-4031-895c-682ef5b6e727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:57.154039 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6bdb4-ea89-4031-895c-682ef5b6e727 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:57.155204 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef6bdb4-ea89-4031-895c-682ef5b6e727 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:57.156932 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef6bdb4-ea89-4031-895c-682ef5b6e727 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:57.158684 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ab4b6f-5d5f-4b2d-b59c-38d1fc243e94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:57.160132 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ab4b6f-5d5f-4b2d-b59c-38d1fc243e94 service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:57.168811 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 809/1599] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:32:57.169727 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6bdb4-ea89-4031-895c-682ef5b6e727 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:57.170112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 791/1600] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:32:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:32:57.170870 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab4b6f-5d5f-4b2d-b59c-38d1fc243e94 service neutron] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:57.179381 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6bdb4-ea89-4031-895c-682ef5b6e727 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:57.180126 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab4b6f-5d5f-4b2d-b59c-38d1fc243e94 service neutron] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:57.184424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cfc81d-ec1d-49ae-b81a-0e0769a2f6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:57.184750 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d826e76e-142e-4500-bff4-adfa864b3820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:57.185300 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfc81d-ec1d-49ae-b81a-0e0769a2f6e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:57.185579 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfc81d-ec1d-49ae-b81a-0e0769a2f6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:57.185691 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826e76e-142e-4500-bff4-adfa864b3820 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:57.185849 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfc81d-ec1d-49ae-b81a-0e0769a2f6e6 admin admin] PATH_INFO: `/v3/projects/66b37250ea6d4c4eb242c27ce9819dfc/users/b2e09539caff4d4698cf29c1ea22d365/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:57.185959 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826e76e-142e-4500-bff4-adfa864b3820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:57.186139 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826e76e-142e-4500-bff4-adfa864b3820 admin admin] PATH_INFO: `/v3/projects/66a98614df914018b7ecfb68afb735c0/users/b5dec80a83aa463b99bd298ab09dfbdb/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:57.203336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d826e76e-142e-4500-bff4-adfa864b3820 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66a98614df914018b7ecfb68afb735c0, user_id=b5dec80a83aa463b99bd298ab09dfbdb, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:57.203608 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cfc81d-ec1d-49ae-b81a-0e0769a2f6e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66b37250ea6d4c4eb242c27ce9819dfc, user_id=b2e09539caff4d4698cf29c1ea22d365, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:57.206544 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cfc81d-ec1d-49ae-b81a-0e0769a2f6e6 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:57.209167 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d826e76e-142e-4500-bff4-adfa864b3820 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:57.369656 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 810/1601] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:32:57 2022] PUT /identity/v3/projects/66b37250ea6d4c4eb242c27ce9819dfc/users/b2e09539caff4d4698cf29c1ea22d365/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:32:57.371344 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d744589-15d8-443c-a179-29d2316ca4eb None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:57.371798 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d744589-15d8-443c-a179-29d2316ca4eb None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:57.373213 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d744589-15d8-443c-a179-29d2316ca4eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:57.374298 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 792/1602] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:32:57 2022] PUT /identity/v3/projects/66a98614df914018b7ecfb68afb735c0/users/b5dec80a83aa463b99bd298ab09dfbdb/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:32:57.375336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d826e76e-142e-4500-bff4-adfa864b3820 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:57.384755 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:32:57.404740 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-4d744589-15d8-443c-a179-29d2316ca4eb None None] MFA Rules not processed for user `0a5b43ce22cc424e87598501e12d256d`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:32:57.406194 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-4d744589-15d8-443c-a179-29d2316ca4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:32:57.406329 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:32:57.444285 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d826e76e-142e-4500-bff4-adfa864b3820 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:57.449220 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d50fc3a-3fba-4052-b212-9ddb60d794d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:57.449887 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50fc3a-3fba-4052-b212-9ddb60d794d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:57.450168 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50fc3a-3fba-4052-b212-9ddb60d794d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:57.450440 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50fc3a-3fba-4052-b212-9ddb60d794d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:57.451172 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d50fc3a-3fba-4052-b212-9ddb60d794d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:57.452793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d50fc3a-3fba-4052-b212-9ddb60d794d2 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:57.464403 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 793/1603] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:32:57.464903 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d50fc3a-3fba-4052-b212-9ddb60d794d2 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:57.473540 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 811/1604] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:32:57 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:32:57.473540 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d744589-15d8-443c-a179-29d2316ca4eb None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:57.477406 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d50fc3a-3fba-4052-b212-9ddb60d794d2 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:57.482679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149120f-4ced-4096-aa7c-0153e4160ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:57.483804 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149120f-4ced-4096-aa7c-0153e4160ff3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:57.484086 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149120f-4ced-4096-aa7c-0153e4160ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:57.484356 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149120f-4ced-4096-aa7c-0153e4160ff3 admin admin] PATH_INFO: `/v3/projects/df4c783aec22478983b30e83a24eae80/users/d0e1c80984ae48868f1321e10c980c89/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:57.486622 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d744589-15d8-443c-a179-29d2316ca4eb None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:57.493838 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5108f1e-4870-43c4-b3db-7d1c5995f15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:57.496852 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5108f1e-4870-43c4-b3db-7d1c5995f15d admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:57.496852 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5108f1e-4870-43c4-b3db-7d1c5995f15d admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:57.496852 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5108f1e-4870-43c4-b3db-7d1c5995f15d admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:57.496852 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5108f1e-4870-43c4-b3db-7d1c5995f15d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:57.497932 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5108f1e-4870-43c4-b3db-7d1c5995f15d admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:57.503328 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a149120f-4ced-4096-aa7c-0153e4160ff3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df4c783aec22478983b30e83a24eae80, user_id=d0e1c80984ae48868f1321e10c980c89, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:57.506489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a149120f-4ced-4096-aa7c-0153e4160ff3 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:57.508946 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 812/1605] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:32:57.510485 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ea18a-7625-491a-8cdd-5e916e1cacdb None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:57.510841 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ea18a-7625-491a-8cdd-5e916e1cacdb None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:57.511493 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ea18a-7625-491a-8cdd-5e916e1cacdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:57.530623 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-f33ea18a-7625-491a-8cdd-5e916e1cacdb None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:32:57.531908 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-f33ea18a-7625-491a-8cdd-5e916e1cacdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:32:57.543812 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 813/1606] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:32:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:32:57.543812 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa869f6-9846-4c49-a3e2-b37f4aed32fd None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:57.543812 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa869f6-9846-4c49-a3e2-b37f4aed32fd None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:57.543812 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa869f6-9846-4c49-a3e2-b37f4aed32fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:57.566188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-8fa869f6-9846-4c49-a3e2-b37f4aed32fd None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:32:57.567161 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-8fa869f6-9846-4c49-a3e2-b37f4aed32fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:32:57.574995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 814/1607] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:32:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:32:57.576466 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa869f6-9846-4c49-a3e2-b37f4aed32fd None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:57.587194 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa869f6-9846-4c49-a3e2-b37f4aed32fd None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:57.591828 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f055d-bde7-4694-9a02-1d93927ba48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:57.592780 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f055d-bde7-4694-9a02-1d93927ba48c admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:57.593132 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f055d-bde7-4694-9a02-1d93927ba48c admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:57.593476 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f055d-bde7-4694-9a02-1d93927ba48c admin admin] PATH_INFO: `/v3/projects/d393195d103b48a1b8d17fc56b3ad594/users/ee2aa825da1542d7a2f1355900e65173/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:57.609476 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1f055d-bde7-4694-9a02-1d93927ba48c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d393195d103b48a1b8d17fc56b3ad594, user_id=ee2aa825da1542d7a2f1355900e65173, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:57.612435 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1f055d-bde7-4694-9a02-1d93927ba48c admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:57.627945 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 794/1608] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:32:57 2022] PUT /identity/v3/projects/df4c783aec22478983b30e83a24eae80/users/d0e1c80984ae48868f1321e10c980c89/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:32:57.628755 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149120f-4ced-4096-aa7c-0153e4160ff3 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:57.700523 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149120f-4ced-4096-aa7c-0153e4160ff3 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:57.705162 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90129867-0ef1-4447-91d5-c89b489034f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:57.705814 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90129867-0ef1-4447-91d5-c89b489034f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:57.706096 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90129867-0ef1-4447-91d5-c89b489034f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:57.706370 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90129867-0ef1-4447-91d5-c89b489034f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:57.720344 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 815/1609] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:32:57 2022] PUT /identity/v3/projects/d393195d103b48a1b8d17fc56b3ad594/users/ee2aa825da1542d7a2f1355900e65173/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:32:57.721541 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f055d-bde7-4694-9a02-1d93927ba48c admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:57.786152 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90129867-0ef1-4447-91d5-c89b489034f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:57.787776 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90129867-0ef1-4447-91d5-c89b489034f8 service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:57.797347 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 795/1610] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:32:57 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:32:57.797909 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90129867-0ef1-4447-91d5-c89b489034f8 service cinder] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:57.800735 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f055d-bde7-4694-9a02-1d93927ba48c admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:57.812084 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34ab4f-423c-4343-a56e-355fb297cdf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:57.813106 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34ab4f-423c-4343-a56e-355fb297cdf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:57.815161 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34ab4f-423c-4343-a56e-355fb297cdf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:57.815548 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34ab4f-423c-4343-a56e-355fb297cdf3 admin admin] PATH_INFO: `/v3/projects/76fc7cc4cb0f46b6bc9d1994c5bf8310/users/cb4114f4d4db425ca547a7778bcc4bc7/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:57.842152 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f34ab4f-423c-4343-a56e-355fb297cdf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76fc7cc4cb0f46b6bc9d1994c5bf8310, user_id=cb4114f4d4db425ca547a7778bcc4bc7, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:57.844927 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f34ab4f-423c-4343-a56e-355fb297cdf3 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:57.862870 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90129867-0ef1-4447-91d5-c89b489034f8 service cinder] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:57.866478 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:32:57.914519 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62fcd4e-9e58-4776-996b-4790b1a9c148 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:57.915174 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62fcd4e-9e58-4776-996b-4790b1a9c148 service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:57.915552 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62fcd4e-9e58-4776-996b-4790b1a9c148 service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:57.915893 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62fcd4e-9e58-4776-996b-4790b1a9c148 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:57.921011 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 816/1611] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:32:57 2022] PUT /identity/v3/projects/76fc7cc4cb0f46b6bc9d1994c5bf8310/users/cb4114f4d4db425ca547a7778bcc4bc7/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:32:57.925198 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34ab4f-423c-4343-a56e-355fb297cdf3 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:57.934289 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62fcd4e-9e58-4776-996b-4790b1a9c148 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:57.935417 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62fcd4e-9e58-4776-996b-4790b1a9c148 service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:57.946757 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 796/1612] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:32:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:32:57.948401 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839c7da-d274-414d-8b91-a48d63c5a734 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:57.948687 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839c7da-d274-414d-8b91-a48d63c5a734 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:57.948961 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839c7da-d274-414d-8b91-a48d63c5a734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:57.984737 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-e839c7da-d274-414d-8b91-a48d63c5a734 None None] MFA Rules not processed for user `d0e1c80984ae48868f1321e10c980c89`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:32:57.986806 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-e839c7da-d274-414d-8b91-a48d63c5a734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:32:58.010985 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:32:58.045077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34ab4f-423c-4343-a56e-355fb297cdf3 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:58.050928 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47648895-0294-4121-8ac4-d0c3f2705728 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:58.051757 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47648895-0294-4121-8ac4-d0c3f2705728 service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:58.052142 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47648895-0294-4121-8ac4-d0c3f2705728 service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:58.052610 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47648895-0294-4121-8ac4-d0c3f2705728 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:58.112130 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 797/1613] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:32:57 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:32:58.112943 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a622b2-86f0-4ba0-b845-bd4e02abba17 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:58.113940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a622b2-86f0-4ba0-b845-bd4e02abba17 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:58.116309 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a622b2-86f0-4ba0-b845-bd4e02abba17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:58.168942 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-40a622b2-86f0-4ba0-b845-bd4e02abba17 None None] MFA Rules not processed for user `ee2aa825da1542d7a2f1355900e65173`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:32:58.172784 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-40a622b2-86f0-4ba0-b845-bd4e02abba17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:32:58.182738 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47648895-0294-4121-8ac4-d0c3f2705728 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:58.184602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47648895-0294-4121-8ac4-d0c3f2705728 service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:58.199185 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 817/1614] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:32:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:32:58.202513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a0383-85ce-401d-a098-745103e50347 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:58.202513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a0383-85ce-401d-a098-745103e50347 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:58.202513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a0383-85ce-401d-a098-745103e50347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:58.225173 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-3d9a0383-85ce-401d-a098-745103e50347 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:32:58.226893 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-3d9a0383-85ce-401d-a098-745103e50347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:32:58.237146 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 818/1615] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:32:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:32:58.240477 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a0383-85ce-401d-a098-745103e50347 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:58.250302 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a0383-85ce-401d-a098-745103e50347 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:58.266063 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32351cc9-593a-42ba-81bb-4771ed82589f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:58.268263 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32351cc9-593a-42ba-81bb-4771ed82589f admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:58.269773 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32351cc9-593a-42ba-81bb-4771ed82589f admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:58.270128 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32351cc9-593a-42ba-81bb-4771ed82589f admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:58.270932 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32351cc9-593a-42ba-81bb-4771ed82589f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:58.273816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32351cc9-593a-42ba-81bb-4771ed82589f admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:58.307717 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 798/1616] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:32:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:32:58.308507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a622b2-86f0-4ba0-b845-bd4e02abba17 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:58.318416 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a622b2-86f0-4ba0-b845-bd4e02abba17 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:58.323394 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e3a26b-e628-46fc-91c9-87d62f776837 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:58.323394 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e3a26b-e628-46fc-91c9-87d62f776837 service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:58.323631 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e3a26b-e628-46fc-91c9-87d62f776837 service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:58.323723 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e3a26b-e628-46fc-91c9-87d62f776837 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:58.340185 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e3a26b-e628-46fc-91c9-87d62f776837 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:58.342163 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e3a26b-e628-46fc-91c9-87d62f776837 service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:58.359888 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 799/1617] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:32:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:32:58.362388 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a5bf5-7079-4c73-b3ab-85d024c6e0a9 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:58.362675 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a5bf5-7079-4c73-b3ab-85d024c6e0a9 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:58.362956 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a5bf5-7079-4c73-b3ab-85d024c6e0a9 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:58.364574 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 800/1618] 10.222.0.25 () {54 vars in 961 bytes} [Thu Jul 7 15:32:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:32:58.370047 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79248b2-aba2-4e79-9df4-15e78fb9c620 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:58.370525 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79248b2-aba2-4e79-9df4-15e78fb9c620 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:58.370883 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79248b2-aba2-4e79-9df4-15e78fb9c620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:58.387463 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 819/1619] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:32:58 2022] POST /identity/v3/projects => generated 365 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:32:58.390867 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32351cc9-593a-42ba-81bb-4771ed82589f admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:58.411995 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-c79248b2-aba2-4e79-9df4-15e78fb9c620 None None] MFA Rules not processed for user `de1632555cb345f2b9213da1925c02f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:32:58.416202 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-c79248b2-aba2-4e79-9df4-15e78fb9c620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:32:58.447160 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:32:58.504401 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32351cc9-593a-42ba-81bb-4771ed82589f admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:58.513225 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beca3b5-088b-4760-948f-eb0d9809ba88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:58.525785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beca3b5-088b-4760-948f-eb0d9809ba88 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:58.526474 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beca3b5-088b-4760-948f-eb0d9809ba88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:58.526858 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beca3b5-088b-4760-948f-eb0d9809ba88 admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:58.527740 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beca3b5-088b-4760-948f-eb0d9809ba88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:58.532333 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beca3b5-088b-4760-948f-eb0d9809ba88 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:58.552047 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 801/1620] 10.222.0.25 () {58 vars in 1058 bytes} [Thu Jul 7 15:32:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:32:58.560721 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79248b2-aba2-4e79-9df4-15e78fb9c620 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:58.574414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79248b2-aba2-4e79-9df4-15e78fb9c620 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:58.585737 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947e7e7-d567-49b8-af3d-e90ee5997e4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:58.586638 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947e7e7-d567-49b8-af3d-e90ee5997e4a service nova] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:58.587247 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947e7e7-d567-49b8-af3d-e90ee5997e4a service nova] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:58.587661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947e7e7-d567-49b8-af3d-e90ee5997e4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:58.624092 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 820/1621] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:32:58 2022] POST /identity/v3/users => generated 289 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:32:58.628103 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beca3b5-088b-4760-948f-eb0d9809ba88 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:58.643806 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beca3b5-088b-4760-948f-eb0d9809ba88 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:58.669678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7972e0d2-3819-4c62-9cb7-572aafbf99be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:58.670418 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7972e0d2-3819-4c62-9cb7-572aafbf99be admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:58.670732 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7972e0d2-3819-4c62-9cb7-572aafbf99be admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:58.671053 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7972e0d2-3819-4c62-9cb7-572aafbf99be admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:58.672056 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7972e0d2-3819-4c62-9cb7-572aafbf99be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:58.673720 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e947e7e7-d567-49b8-af3d-e90ee5997e4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:58.673908 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7972e0d2-3819-4c62-9cb7-572aafbf99be admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:58.675940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e947e7e7-d567-49b8-af3d-e90ee5997e4a service nova] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:58.687106 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 802/1622] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:32:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:32:58.690906 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 821/1623] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:32:58 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:32:58.692810 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947e7e7-d567-49b8-af3d-e90ee5997e4a service nova] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:58.713608 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947e7e7-d567-49b8-af3d-e90ee5997e4a service nova] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:58.719297 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbea0ca-2ee5-4b45-ae18-9f6ecfeca8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:58.720794 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbea0ca-2ee5-4b45-ae18-9f6ecfeca8b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:58.725529 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbea0ca-2ee5-4b45-ae18-9f6ecfeca8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:58.725529 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbea0ca-2ee5-4b45-ae18-9f6ecfeca8b4 admin admin] PATH_INFO: `/v3/projects/6d85693835db4b36b8792c12ae7f76d0/users/6715e3caf1b145e98be4ce66f74cd0ab/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:58.742749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbea0ca-2ee5-4b45-ae18-9f6ecfeca8b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d85693835db4b36b8792c12ae7f76d0, user_id=6715e3caf1b145e98be4ce66f74cd0ab, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:58.746571 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbea0ca-2ee5-4b45-ae18-9f6ecfeca8b4 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:58.860150 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 803/1624] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:32:58 2022] PUT /identity/v3/projects/6d85693835db4b36b8792c12ae7f76d0/users/6715e3caf1b145e98be4ce66f74cd0ab/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:32:58.864184 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7972e0d2-3819-4c62-9cb7-572aafbf99be admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:58.878699 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7972e0d2-3819-4c62-9cb7-572aafbf99be admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:58.935957 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:32:59.013793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11342f2a-1b86-4b45-91a8-c0baff9519b8 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:59.014018 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11342f2a-1b86-4b45-91a8-c0baff9519b8 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:59.014316 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11342f2a-1b86-4b45-91a8-c0baff9519b8 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:59.015864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 804/1625] 10.222.0.25 () {54 vars in 912 bytes} [Thu Jul 7 15:32:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:32:59.023605 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a65acd-6782-490e-a7b2-cb2dd96bc59a None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:59.023922 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a65acd-6782-490e-a7b2-cb2dd96bc59a None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:59.024245 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a65acd-6782-490e-a7b2-cb2dd96bc59a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:59.044899 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe4768-39b5-40fb-8a1a-922e374bf743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:59.046353 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe4768-39b5-40fb-8a1a-922e374bf743 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:59.046640 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe4768-39b5-40fb-8a1a-922e374bf743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:59.046913 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe4768-39b5-40fb-8a1a-922e374bf743 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:59.047735 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fe4768-39b5-40fb-8a1a-922e374bf743 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:59.049751 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fe4768-39b5-40fb-8a1a-922e374bf743 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:59.055239 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-a3a65acd-6782-490e-a7b2-cb2dd96bc59a None None] MFA Rules not processed for user `de1632555cb345f2b9213da1925c02f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:32:59.056672 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-a3a65acd-6782-490e-a7b2-cb2dd96bc59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:32:59.062889 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 822/1626] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:32:58 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:32:59.070056 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe4768-39b5-40fb-8a1a-922e374bf743 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:59.084968 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe4768-39b5-40fb-8a1a-922e374bf743 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:59.090622 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7d6eef-1f3f-42b8-a59e-a7d462f96a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:59.091685 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d6eef-1f3f-42b8-a59e-a7d462f96a83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:59.092080 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d6eef-1f3f-42b8-a59e-a7d462f96a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:59.092469 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d6eef-1f3f-42b8-a59e-a7d462f96a83 admin admin] PATH_INFO: `/v3/projects/6d85693835db4b36b8792c12ae7f76d0/users/6715e3caf1b145e98be4ce66f74cd0ab/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:59.094984 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7d6eef-1f3f-42b8-a59e-a7d462f96a83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d85693835db4b36b8792c12ae7f76d0, user_id=6715e3caf1b145e98be4ce66f74cd0ab, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:59.098629 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7d6eef-1f3f-42b8-a59e-a7d462f96a83 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:59.123236 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 823/1627] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:32:59 2022] PUT /identity/v3/projects/6d85693835db4b36b8792c12ae7f76d0/users/6715e3caf1b145e98be4ce66f74cd0ab/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:32:59.131730 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fe46a-989e-4117-9e51-2142b94bb1f2 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:59.131730 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fe46a-989e-4117-9e51-2142b94bb1f2 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:59.131730 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fe46a-989e-4117-9e51-2142b94bb1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:59.165965 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-026fe46a-989e-4117-9e51-2142b94bb1f2 None None] MFA Rules not processed for user `6715e3caf1b145e98be4ce66f74cd0ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:32:59.167748 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-026fe46a-989e-4117-9e51-2142b94bb1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:32:59.182241 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 805/1628] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Jul 7 15:32:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:32:59.195853 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a65acd-6782-490e-a7b2-cb2dd96bc59a None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:59.218973 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:32:59.313406 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a65acd-6782-490e-a7b2-cb2dd96bc59a None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:59.338307 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 824/1629] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:32:59 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:32:59.362321 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026fe46a-989e-4117-9e51-2142b94bb1f2 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:59.422695 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36546e3-fdfe-464a-afe5-7de8f32a7fc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:59.423345 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36546e3-fdfe-464a-afe5-7de8f32a7fc0 service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:59.423692 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36546e3-fdfe-464a-afe5-7de8f32a7fc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:59.423901 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36546e3-fdfe-464a-afe5-7de8f32a7fc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:59.457411 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026fe46a-989e-4117-9e51-2142b94bb1f2 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:59.465078 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2050f202-3738-400c-9dd0-4f35a3c64dcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:59.469248 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050f202-3738-400c-9dd0-4f35a3c64dcf service cinder] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:59.469248 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050f202-3738-400c-9dd0-4f35a3c64dcf service cinder] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:59.469248 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050f202-3738-400c-9dd0-4f35a3c64dcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:59.483963 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2050f202-3738-400c-9dd0-4f35a3c64dcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:59.485439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2050f202-3738-400c-9dd0-4f35a3c64dcf service cinder] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:59.510217 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 825/1630] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:32:59 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:32:59.541562 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36546e3-fdfe-464a-afe5-7de8f32a7fc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:59.543061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36546e3-fdfe-464a-afe5-7de8f32a7fc0 service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:59.559075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 806/1631] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:32:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:32:59.823124 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2050f202-3738-400c-9dd0-4f35a3c64dcf service cinder] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:32:59.835221 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2050f202-3738-400c-9dd0-4f35a3c64dcf service cinder] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:32:59.840617 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff71a30-c14b-4dba-9986-75f93cf8bdd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:32:59.841335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff71a30-c14b-4dba-9986-75f93cf8bdd9 service cinder] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:32:59.841890 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff71a30-c14b-4dba-9986-75f93cf8bdd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:32:59.842264 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff71a30-c14b-4dba-9986-75f93cf8bdd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:32:59.906746 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff71a30-c14b-4dba-9986-75f93cf8bdd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:32:59.910653 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff71a30-c14b-4dba-9986-75f93cf8bdd9 service cinder] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:32:59.915049 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:32:59.923117 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 826/1632] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:32:59 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:33:00.635478 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0104bb4-b1ef-498a-be51-27c30bfff7aa None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:00.644329 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0104bb4-b1ef-498a-be51-27c30bfff7aa None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:00.644329 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0104bb4-b1ef-498a-be51-27c30bfff7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:00.726463 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-a0104bb4-b1ef-498a-be51-27c30bfff7aa None None] MFA Rules not processed for user `cb4114f4d4db425ca547a7778bcc4bc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:00.726463 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-a0104bb4-b1ef-498a-be51-27c30bfff7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:00.873139 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:00.887211 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 807/1633] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:00 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:00.891427 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff71a30-c14b-4dba-9986-75f93cf8bdd9 service cinder] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:01.008661 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff71a30-c14b-4dba-9986-75f93cf8bdd9 service cinder] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:01.013555 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a5a1d2-2cc7-4575-a528-fe72f0392bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:01.014146 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5a1d2-2cc7-4575-a528-fe72f0392bc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:01.014506 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5a1d2-2cc7-4575-a528-fe72f0392bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:01.014779 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5a1d2-2cc7-4575-a528-fe72f0392bc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:01.016029 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a5a1d2-2cc7-4575-a528-fe72f0392bc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:01.018990 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a5a1d2-2cc7-4575-a528-fe72f0392bc5 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:01.134606 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6896df9-b97a-42d4-a482-3fd654cb2fb0 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:01.134606 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6896df9-b97a-42d4-a482-3fd654cb2fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:01.134606 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6896df9-b97a-42d4-a482-3fd654cb2fb0 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:01.134606 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 808/1634] 10.222.0.25 () {54 vars in 961 bytes} [Thu Jul 7 15:33:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:33:01.134606 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad011b2-65cb-486f-ba87-85152c8a9e6d None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:01.134606 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad011b2-65cb-486f-ba87-85152c8a9e6d None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:01.138522 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad011b2-65cb-486f-ba87-85152c8a9e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:01.138522 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-4ad011b2-65cb-486f-ba87-85152c8a9e6d None None] MFA Rules not processed for user `de1632555cb345f2b9213da1925c02f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:01.138522 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-4ad011b2-65cb-486f-ba87-85152c8a9e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:01.154711 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 809/1635] 10.222.0.25 () {58 vars in 1058 bytes} [Thu Jul 7 15:33:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:01.161389 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 827/1636] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:33:00 2022] POST /identity/v3/projects => generated 405 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:33:01.162529 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597a9f1-1dc4-44bb-b9b9-390a87747688 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:01.162823 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597a9f1-1dc4-44bb-b9b9-390a87747688 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:01.163106 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597a9f1-1dc4-44bb-b9b9-390a87747688 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:01.164840 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 810/1637] 10.222.0.25 () {54 vars in 912 bytes} [Thu Jul 7 15:33:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:33:01.165095 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a5a1d2-2cc7-4575-a528-fe72f0392bc5 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:01.174265 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690e8b9-bd22-4b20-9621-21a5476fa2ac None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:01.174669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690e8b9-bd22-4b20-9621-21a5476fa2ac None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:01.175049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690e8b9-bd22-4b20-9621-21a5476fa2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:01.215658 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-e690e8b9-bd22-4b20-9621-21a5476fa2ac None None] MFA Rules not processed for user `de1632555cb345f2b9213da1925c02f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:01.217065 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-e690e8b9-bd22-4b20-9621-21a5476fa2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:01.303181 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a5a1d2-2cc7-4575-a528-fe72f0392bc5 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:01.315960 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b3ca67-9496-4dd2-a0d1-555b3047dc98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:01.316943 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3ca67-9496-4dd2-a0d1-555b3047dc98 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:01.317344 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3ca67-9496-4dd2-a0d1-555b3047dc98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:01.318297 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3ca67-9496-4dd2-a0d1-555b3047dc98 admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:01.318574 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b3ca67-9496-4dd2-a0d1-555b3047dc98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:01.321184 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b3ca67-9496-4dd2-a0d1-555b3047dc98 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:01.348732 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 811/1638] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Jul 7 15:33:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:01.362001 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e690e8b9-bd22-4b20-9621-21a5476fa2ac None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:01.435339 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 828/1639] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:33:01 2022] POST /identity/v3/users => generated 309 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:33:01.440678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b3ca67-9496-4dd2-a0d1-555b3047dc98 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:01.468238 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b3ca67-9496-4dd2-a0d1-555b3047dc98 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:01.472213 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e690e8b9-bd22-4b20-9621-21a5476fa2ac None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:01.481164 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa254625-441d-49b5-be38-c5284efbf8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:01.483710 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4544672c-0289-411e-9927-f91d0401b63d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:01.484476 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa254625-441d-49b5-be38-c5284efbf8a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:01.485060 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa254625-441d-49b5-be38-c5284efbf8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:01.486373 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa254625-441d-49b5-be38-c5284efbf8a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:01.488738 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa254625-441d-49b5-be38-c5284efbf8a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:01.491127 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544672c-0289-411e-9927-f91d0401b63d service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:01.491127 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544672c-0289-411e-9927-f91d0401b63d service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:01.491741 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544672c-0289-411e-9927-f91d0401b63d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:01.492758 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa254625-441d-49b5-be38-c5284efbf8a7 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:01.525619 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4544672c-0289-411e-9927-f91d0401b63d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:01.529649 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4544672c-0289-411e-9927-f91d0401b63d service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:01.530900 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 829/1640] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:01 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:33:01.535181 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa254625-441d-49b5-be38-c5284efbf8a7 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:01.552421 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa254625-441d-49b5-be38-c5284efbf8a7 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:01.564955 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f86c4ab-9fd4-4b6e-b502-674579b63355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:01.565121 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 812/1641] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:33:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 15:33:01.565874 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86c4ab-9fd4-4b6e-b502-674579b63355 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:01.566237 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86c4ab-9fd4-4b6e-b502-674579b63355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:01.566490 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86c4ab-9fd4-4b6e-b502-674579b63355 admin admin] PATH_INFO: `/v3/projects/d8e382508a4e4918ade240f2cefd9865/users/6e2125ee287a40acbb9d30b8bfeadb08/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:01.633507 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f86c4ab-9fd4-4b6e-b502-674579b63355 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8e382508a4e4918ade240f2cefd9865, user_id=6e2125ee287a40acbb9d30b8bfeadb08, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:01.633507 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f86c4ab-9fd4-4b6e-b502-674579b63355 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:01.741326 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 830/1642] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:33:01 2022] PUT /identity/v3/projects/d8e382508a4e4918ade240f2cefd9865/users/6e2125ee287a40acbb9d30b8bfeadb08/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:33:01.744476 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4544672c-0289-411e-9927-f91d0401b63d service cinder] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:01.774192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4544672c-0289-411e-9927-f91d0401b63d service cinder] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:01.934177 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075a416-4fff-4e87-8fdc-31a6b98da0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:01.935121 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075a416-4fff-4e87-8fdc-31a6b98da0f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:01.935508 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075a416-4fff-4e87-8fdc-31a6b98da0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:01.935913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075a416-4fff-4e87-8fdc-31a6b98da0f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:01.937468 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f075a416-4fff-4e87-8fdc-31a6b98da0f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:01.942853 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f075a416-4fff-4e87-8fdc-31a6b98da0f2 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:01.974037 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 813/1643] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:01 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:33:01.975556 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f86c4ab-9fd4-4b6e-b502-674579b63355 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:01.995266 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f86c4ab-9fd4-4b6e-b502-674579b63355 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:02.006507 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b6e451-f50d-43d1-aefa-e12fb2b0404f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:02.007419 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b6e451-f50d-43d1-aefa-e12fb2b0404f admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:02.236396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b6e451-f50d-43d1-aefa-e12fb2b0404f admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:02.236396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b6e451-f50d-43d1-aefa-e12fb2b0404f admin admin] PATH_INFO: `/v3/projects/d8e382508a4e4918ade240f2cefd9865/users/6e2125ee287a40acbb9d30b8bfeadb08/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:02.236396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b6e451-f50d-43d1-aefa-e12fb2b0404f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8e382508a4e4918ade240f2cefd9865, user_id=6e2125ee287a40acbb9d30b8bfeadb08, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:02.236396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b6e451-f50d-43d1-aefa-e12fb2b0404f admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:02.236396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 831/1644] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:33:01 2022] PUT /identity/v3/projects/d8e382508a4e4918ade240f2cefd9865/users/6e2125ee287a40acbb9d30b8bfeadb08/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:03.915288 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a1400-c823-44c6-8f87-d20f8a8a9d49 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:03.921023 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a1400-c823-44c6-8f87-d20f8a8a9d49 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:03.921023 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a1400-c823-44c6-8f87-d20f8a8a9d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:03.972795 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-1b4a1400-c823-44c6-8f87-d20f8a8a9d49 None None] MFA Rules not processed for user `6e2125ee287a40acbb9d30b8bfeadb08`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:03.984431 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-1b4a1400-c823-44c6-8f87-d20f8a8a9d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:04.023077 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:04.274706 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 814/1645] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:03 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:06.412816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417729e4-e53f-42ea-bb93-e3c64bf4d1c7 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:06.418395 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417729e4-e53f-42ea-bb93-e3c64bf4d1c7 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:06.418395 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417729e4-e53f-42ea-bb93-e3c64bf4d1c7 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:06.418395 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 832/1646] 10.222.0.25 () {54 vars in 921 bytes} [Thu Jul 7 15:33:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:33:06.441694 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57737a63-f629-4c8c-b349-b87c0f9ced0c None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:06.447786 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57737a63-f629-4c8c-b349-b87c0f9ced0c None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:06.447786 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57737a63-f629-4c8c-b349-b87c0f9ced0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:06.490500 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-57737a63-f629-4c8c-b349-b87c0f9ced0c None None] MFA Rules not processed for user `755c2782cfb042f89950b8fc862e4fa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:06.492033 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-57737a63-f629-4c8c-b349-b87c0f9ced0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:06.689929 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 815/1647] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Jul 7 15:33:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:06.777875 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867f2f2-dde1-4491-9cf1-06806b2c2f18 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:06.778508 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867f2f2-dde1-4491-9cf1-06806b2c2f18 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:06.778922 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867f2f2-dde1-4491-9cf1-06806b2c2f18 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:06.780486 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 833/1648] 10.222.0.25 () {54 vars in 960 bytes} [Thu Jul 7 15:33:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:33:06.791452 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1974b71-eb4c-40e3-b94c-249fddc8871d None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:06.791829 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1974b71-eb4c-40e3-b94c-249fddc8871d None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:06.792108 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1974b71-eb4c-40e3-b94c-249fddc8871d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:06.832992 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-b1974b71-eb4c-40e3-b94c-249fddc8871d None None] MFA Rules not processed for user `755c2782cfb042f89950b8fc862e4fa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:06.834882 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-b1974b71-eb4c-40e3-b94c-249fddc8871d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:06.846821 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 816/1649] 10.222.0.25 () {58 vars in 1057 bytes} [Thu Jul 7 15:33:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:06.857440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8867f2f2-dde1-4491-9cf1-06806b2c2f18 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:06.872975 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:06.887591 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8867f2f2-dde1-4491-9cf1-06806b2c2f18 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:06.899069 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9bdfc8-14f5-478e-b555-56c18a9b5ef7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '755c2782cfb042f89950b8fc862e4fa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:06.899858 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9bdfc8-14f5-478e-b555-56c18a9b5ef7 service placement] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:06.900298 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9bdfc8-14f5-478e-b555-56c18a9b5ef7 service placement] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:06.900863 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9bdfc8-14f5-478e-b555-56c18a9b5ef7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:06.913778 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9bdfc8-14f5-478e-b555-56c18a9b5ef7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:06.916670 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9bdfc8-14f5-478e-b555-56c18a9b5ef7 service placement] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:06.933876 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 834/1650] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:33:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:33:07.118827 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1974b71-eb4c-40e3-b94c-249fddc8871d None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:07.280384 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1974b71-eb4c-40e3-b94c-249fddc8871d None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:07.295141 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003a06e0-07ca-4616-95df-ff43f36913ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:07.296227 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a06e0-07ca-4616-95df-ff43f36913ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:07.296645 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a06e0-07ca-4616-95df-ff43f36913ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:07.297032 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a06e0-07ca-4616-95df-ff43f36913ab admin admin] PATH_INFO: `/v3/users/d0e1c80984ae48868f1321e10c980c89` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:07.299242 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003a06e0-07ca-4616-95df-ff43f36913ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0e1c80984ae48868f1321e10c980c89)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:07.300480 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: WARNING py.warnings [None req-003a06e0-07ca-4616-95df-ff43f36913ab admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 07 15:33:07.300480 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: warnings.warn(msg) Jul 07 15:33:07.300480 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: Jul 07 15:33:07.301932 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003a06e0-07ca-4616-95df-ff43f36913ab admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:07.640744 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:07.659786 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-003a06e0-07ca-4616-95df-ff43f36913ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0e1c80984ae48868f1321e10c980c89'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:07.690163 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-003a06e0-07ca-4616-95df-ff43f36913ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0e1c80984ae48868f1321e10c980c89'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:07.764905 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-003a06e0-07ca-4616-95df-ff43f36913ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0e1c80984ae48868f1321e10c980c89'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:07.765939 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-003a06e0-07ca-4616-95df-ff43f36913ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0e1c80984ae48868f1321e10c980c89'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:07.871311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-003a06e0-07ca-4616-95df-ff43f36913ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0e1c80984ae48868f1321e10c980c89'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:07.880210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 817/1651] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:33:07 2022] DELETE /identity/v3/users/d0e1c80984ae48868f1321e10c980c89 => generated 0 bytes in 762 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:07.884888 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea549b96-46e5-4388-93b0-9d204b0dcd2a None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:07.885406 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea549b96-46e5-4388-93b0-9d204b0dcd2a None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:07.885919 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea549b96-46e5-4388-93b0-9d204b0dcd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:07.929879 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-ea549b96-46e5-4388-93b0-9d204b0dcd2a None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:07.932429 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-ea549b96-46e5-4388-93b0-9d204b0dcd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:08.143172 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 835/1652] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:08.496602 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003a06e0-07ca-4616-95df-ff43f36913ab admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:08.498239 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-003a06e0-07ca-4616-95df-ff43f36913ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:08.746047 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003a06e0-07ca-4616-95df-ff43f36913ab admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:08.758333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15fe2b0-6c89-43ba-9364-9e073bb0d1d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:08.759383 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fe2b0-6c89-43ba-9364-9e073bb0d1d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:08.759698 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fe2b0-6c89-43ba-9364-9e073bb0d1d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:08.759978 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fe2b0-6c89-43ba-9364-9e073bb0d1d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:08.788451 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15fe2b0-6c89-43ba-9364-9e073bb0d1d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:08.790167 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15fe2b0-6c89-43ba-9364-9e073bb0d1d9 service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:08.818424 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 818/1653] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:33:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:33:10.659013 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea549b96-46e5-4388-93b0-9d204b0dcd2a None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:10.660526 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-ea549b96-46e5-4388-93b0-9d204b0dcd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:10.692719 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea549b96-46e5-4388-93b0-9d204b0dcd2a None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:10.716455 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c9ea5-1f6e-4cb1-83e4-9dbe534a2da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:10.717272 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c9ea5-1f6e-4cb1-83e4-9dbe534a2da7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:10.717559 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c9ea5-1f6e-4cb1-83e4-9dbe534a2da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:10.717835 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c9ea5-1f6e-4cb1-83e4-9dbe534a2da7 admin admin] PATH_INFO: `/v3/users/cb4114f4d4db425ca547a7778bcc4bc7` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:10.738606 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c9ea5-1f6e-4cb1-83e4-9dbe534a2da7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb4114f4d4db425ca547a7778bcc4bc7)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:10.738606 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c9ea5-1f6e-4cb1-83e4-9dbe534a2da7 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:10.911723 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15fe2b0-6c89-43ba-9364-9e073bb0d1d9 service neutron] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:10.915178 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-a15fe2b0-6c89-43ba-9364-9e073bb0d1d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:10.937124 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15fe2b0-6c89-43ba-9364-9e073bb0d1d9 service neutron] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:10.943513 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ce9410-a031-4da5-828d-df6f97e78a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:10.944837 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce9410-a031-4da5-828d-df6f97e78a8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:10.945310 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce9410-a031-4da5-828d-df6f97e78a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:10.945755 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce9410-a031-4da5-828d-df6f97e78a8f admin admin] PATH_INFO: `/v3/users/ee2aa825da1542d7a2f1355900e65173` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:10.952737 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ce9410-a031-4da5-828d-df6f97e78a8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee2aa825da1542d7a2f1355900e65173)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:10.952737 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ce9410-a031-4da5-828d-df6f97e78a8f admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:11.116143 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-799c9ea5-1f6e-4cb1-83e4-9dbe534a2da7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb4114f4d4db425ca547a7778bcc4bc7'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:11.118995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-799c9ea5-1f6e-4cb1-83e4-9dbe534a2da7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb4114f4d4db425ca547a7778bcc4bc7'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:11.145968 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-799c9ea5-1f6e-4cb1-83e4-9dbe534a2da7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb4114f4d4db425ca547a7778bcc4bc7'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:11.147718 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-799c9ea5-1f6e-4cb1-83e4-9dbe534a2da7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb4114f4d4db425ca547a7778bcc4bc7'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:11.183695 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-799c9ea5-1f6e-4cb1-83e4-9dbe534a2da7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb4114f4d4db425ca547a7778bcc4bc7'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:11.213716 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-16ce9410-a031-4da5-828d-df6f97e78a8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee2aa825da1542d7a2f1355900e65173'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:11.223410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-16ce9410-a031-4da5-828d-df6f97e78a8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee2aa825da1542d7a2f1355900e65173'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:11.259032 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-16ce9410-a031-4da5-828d-df6f97e78a8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee2aa825da1542d7a2f1355900e65173'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:11.262648 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-16ce9410-a031-4da5-828d-df6f97e78a8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee2aa825da1542d7a2f1355900e65173'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:11.325664 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 836/1654] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:33:10 2022] DELETE /identity/v3/users/cb4114f4d4db425ca547a7778bcc4bc7 => generated 0 bytes in 664 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:11.326929 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c9ea5-1f6e-4cb1-83e4-9dbe534a2da7 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:11.330309 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-799c9ea5-1f6e-4cb1-83e4-9dbe534a2da7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:11.363214 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-16ce9410-a031-4da5-828d-df6f97e78a8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee2aa825da1542d7a2f1355900e65173'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:11.380026 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 819/1655] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:33:10 2022] DELETE /identity/v3/users/ee2aa825da1542d7a2f1355900e65173 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:11.386666 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff72fd-d21a-47c5-b124-2799c111f7d9 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:11.387304 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff72fd-d21a-47c5-b124-2799c111f7d9 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:11.390105 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff72fd-d21a-47c5-b124-2799c111f7d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:11.450162 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-4fff72fd-d21a-47c5-b124-2799c111f7d9 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:11.454473 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-4fff72fd-d21a-47c5-b124-2799c111f7d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:11.499671 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c9ea5-1f6e-4cb1-83e4-9dbe534a2da7 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:11.517063 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:11.530180 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:11.570934 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 820/1656] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:11.585295 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff72fd-d21a-47c5-b124-2799c111f7d9 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:11.587423 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-4fff72fd-d21a-47c5-b124-2799c111f7d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:11.676632 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da8642a-c836-4f92-a4cc-8a97c1d9e2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:11.677635 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da8642a-c836-4f92-a4cc-8a97c1d9e2b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:11.685444 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da8642a-c836-4f92-a4cc-8a97c1d9e2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:11.685444 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da8642a-c836-4f92-a4cc-8a97c1d9e2b1 admin admin] PATH_INFO: `/v3/projects/df4c783aec22478983b30e83a24eae80` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:11.685444 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da8642a-c836-4f92-a4cc-8a97c1d9e2b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df4c783aec22478983b30e83a24eae80)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:11.685444 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: WARNING py.warnings [None req-0da8642a-c836-4f92-a4cc-8a97c1d9e2b1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 07 15:33:11.685444 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: warnings.warn(msg) Jul 07 15:33:11.685444 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: Jul 07 15:33:11.687349 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da8642a-c836-4f92-a4cc-8a97c1d9e2b1 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:11.690079 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff72fd-d21a-47c5-b124-2799c111f7d9 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:11.694943 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911c18b0-a8a0-4c8f-a9df-ec5e7108168d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:11.696010 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c18b0-a8a0-4c8f-a9df-ec5e7108168d service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:11.696511 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c18b0-a8a0-4c8f-a9df-ec5e7108168d service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:11.696973 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c18b0-a8a0-4c8f-a9df-ec5e7108168d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:11.709645 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911c18b0-a8a0-4c8f-a9df-ec5e7108168d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:11.711386 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911c18b0-a8a0-4c8f-a9df-ec5e7108168d service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:11.725073 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 821/1657] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:33:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:33:11.859254 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-0da8642a-c836-4f92-a4cc-8a97c1d9e2b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df4c783aec22478983b30e83a24eae80', 'request_id': 'req-0da8642a-c836-4f92-a4cc-8a97c1d9e2b1'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:11.868922 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-0da8642a-c836-4f92-a4cc-8a97c1d9e2b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df4c783aec22478983b30e83a24eae80', 'request_id': 'req-0da8642a-c836-4f92-a4cc-8a97c1d9e2b1'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:11.900557 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-0da8642a-c836-4f92-a4cc-8a97c1d9e2b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df4c783aec22478983b30e83a24eae80', 'request_id': 'req-0da8642a-c836-4f92-a4cc-8a97c1d9e2b1'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:12.025122 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-0da8642a-c836-4f92-a4cc-8a97c1d9e2b1 admin admin] The token cache is being invalidate because project df4c783aec22478983b30e83a24eae80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:33:12.025415 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-0da8642a-c836-4f92-a4cc-8a97c1d9e2b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:12.026524 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-0da8642a-c836-4f92-a4cc-8a97c1d9e2b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:12.026748 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd450a-982d-499e-83ba-22b153315386 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:12.027145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd450a-982d-499e-83ba-22b153315386 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:12.027429 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd450a-982d-499e-83ba-22b153315386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:12.034470 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 837/1658] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:33:11 2022] DELETE /identity/v3/projects/df4c783aec22478983b30e83a24eae80 => generated 0 bytes in 705 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:12.042896 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da8642a-c836-4f92-a4cc-8a97c1d9e2b1 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:12.046680 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-0da8642a-c836-4f92-a4cc-8a97c1d9e2b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:12.104638 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-fbcd450a-982d-499e-83ba-22b153315386 None None] MFA Rules not processed for user `b5dec80a83aa463b99bd298ab09dfbdb`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:12.108630 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-fbcd450a-982d-499e-83ba-22b153315386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:12.241090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da8642a-c836-4f92-a4cc-8a97c1d9e2b1 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:12.255998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aaeae4-480a-4b11-ab54-46bec7cbc0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:12.256909 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aaeae4-480a-4b11-ab54-46bec7cbc0dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:12.257194 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aaeae4-480a-4b11-ab54-46bec7cbc0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:12.259433 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aaeae4-480a-4b11-ab54-46bec7cbc0dc admin admin] PATH_INFO: `/v3/users/6715e3caf1b145e98be4ce66f74cd0ab` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:12.262772 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aaeae4-480a-4b11-ab54-46bec7cbc0dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=6715e3caf1b145e98be4ce66f74cd0ab)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:12.267122 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aaeae4-480a-4b11-ab54-46bec7cbc0dc admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:12.315008 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 822/1659] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:12.316245 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcd450a-982d-499e-83ba-22b153315386 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:12.327234 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-fbcd450a-982d-499e-83ba-22b153315386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:12.353796 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcd450a-982d-499e-83ba-22b153315386 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:12.391751 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730fafb-98ae-43e4-93ed-35eed5fb460a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:12.392737 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730fafb-98ae-43e4-93ed-35eed5fb460a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:12.393053 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730fafb-98ae-43e4-93ed-35eed5fb460a admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:12.393361 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730fafb-98ae-43e4-93ed-35eed5fb460a admin admin] PATH_INFO: `/v3/projects/76fc7cc4cb0f46b6bc9d1994c5bf8310` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:12.395854 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f730fafb-98ae-43e4-93ed-35eed5fb460a admin admin] RBAC: Authorizing `identity:delete_project(project_id=76fc7cc4cb0f46b6bc9d1994c5bf8310)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:12.402087 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f730fafb-98ae-43e4-93ed-35eed5fb460a admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:12.547991 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:12.558140 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-f4aaeae4-480a-4b11-ab54-46bec7cbc0dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6715e3caf1b145e98be4ce66f74cd0ab'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:12.559087 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-f4aaeae4-480a-4b11-ab54-46bec7cbc0dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6715e3caf1b145e98be4ce66f74cd0ab'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:12.574041 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-f4aaeae4-480a-4b11-ab54-46bec7cbc0dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6715e3caf1b145e98be4ce66f74cd0ab'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:12.575120 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-f4aaeae4-480a-4b11-ab54-46bec7cbc0dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6715e3caf1b145e98be4ce66f74cd0ab'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:12.597610 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-f4aaeae4-480a-4b11-ab54-46bec7cbc0dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6715e3caf1b145e98be4ce66f74cd0ab'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:12.621180 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-f730fafb-98ae-43e4-93ed-35eed5fb460a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76fc7cc4cb0f46b6bc9d1994c5bf8310', 'request_id': 'req-f730fafb-98ae-43e4-93ed-35eed5fb460a'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:12.629798 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-f730fafb-98ae-43e4-93ed-35eed5fb460a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76fc7cc4cb0f46b6bc9d1994c5bf8310', 'request_id': 'req-f730fafb-98ae-43e4-93ed-35eed5fb460a'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:12.632583 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:12.638799 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-f730fafb-98ae-43e4-93ed-35eed5fb460a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76fc7cc4cb0f46b6bc9d1994c5bf8310', 'request_id': 'req-f730fafb-98ae-43e4-93ed-35eed5fb460a'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:12.720351 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 838/1660] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:33:12 2022] DELETE /identity/v3/users/6715e3caf1b145e98be4ce66f74cd0ab => generated 0 bytes in 680 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:12.721939 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aaeae4-480a-4b11-ab54-46bec7cbc0dc admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:12.725932 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-f4aaeae4-480a-4b11-ab54-46bec7cbc0dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:12.810172 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-f730fafb-98ae-43e4-93ed-35eed5fb460a admin admin] The token cache is being invalidate because project 76fc7cc4cb0f46b6bc9d1994c5bf8310 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:33:12.810998 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-f730fafb-98ae-43e4-93ed-35eed5fb460a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:12.811433 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-f730fafb-98ae-43e4-93ed-35eed5fb460a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:12.814283 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 823/1661] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:33:12 2022] DELETE /identity/v3/projects/76fc7cc4cb0f46b6bc9d1994c5bf8310 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:12.831743 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa2422-91f7-4795-a5ec-35f92f791adb None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:12.832133 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa2422-91f7-4795-a5ec-35f92f791adb None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:12.840081 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa2422-91f7-4795-a5ec-35f92f791adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:12.903730 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-52fa2422-91f7-4795-a5ec-35f92f791adb None None] MFA Rules not processed for user `332b1c9c68db4175a1383bb588b5e822`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:12.905462 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-52fa2422-91f7-4795-a5ec-35f92f791adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:12.982666 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aaeae4-480a-4b11-ab54-46bec7cbc0dc admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:13.020745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80af59-3ec6-4f05-bf44-fa4a8fcb40bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:13.022892 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80af59-3ec6-4f05-bf44-fa4a8fcb40bc service nova] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:13.022892 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80af59-3ec6-4f05-bf44-fa4a8fcb40bc service nova] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:13.022892 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80af59-3ec6-4f05-bf44-fa4a8fcb40bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:13.028775 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-1f80af59-3ec6-4f05-bf44-fa4a8fcb40bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:13.058034 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 824/1662] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:13.058034 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66d0b7-c958-40b0-a87b-be611a59f0c7 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:13.058034 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66d0b7-c958-40b0-a87b-be611a59f0c7 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:13.058034 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66d0b7-c958-40b0-a87b-be611a59f0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:13.109055 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-ee66d0b7-c958-40b0-a87b-be611a59f0c7 None None] MFA Rules not processed for user `2033ad40d45747aa8294e4feb5a6cfe0`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:13.111845 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-ee66d0b7-c958-40b0-a87b-be611a59f0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:13.179061 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f80af59-3ec6-4f05-bf44-fa4a8fcb40bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:13.182053 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f80af59-3ec6-4f05-bf44-fa4a8fcb40bc service nova] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:13.211236 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 839/1663] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:33:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:33:13.212139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80af59-3ec6-4f05-bf44-fa4a8fcb40bc service nova] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:13.213670 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-1f80af59-3ec6-4f05-bf44-fa4a8fcb40bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:13.294154 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 825/1664] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:13.294154 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01059e-b952-4ec9-9952-489c06a62e4e None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:13.294154 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01059e-b952-4ec9-9952-489c06a62e4e None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:13.294154 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01059e-b952-4ec9-9952-489c06a62e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:13.326486 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80af59-3ec6-4f05-bf44-fa4a8fcb40bc service nova] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:13.332726 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5797e352-8645-4f03-b47a-77193f51f1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:13.333850 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797e352-8645-4f03-b47a-77193f51f1a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:13.334279 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797e352-8645-4f03-b47a-77193f51f1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:13.334686 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797e352-8645-4f03-b47a-77193f51f1a0 admin admin] PATH_INFO: `/v3/users/6e2125ee287a40acbb9d30b8bfeadb08` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:13.336874 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5797e352-8645-4f03-b47a-77193f51f1a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e2125ee287a40acbb9d30b8bfeadb08)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:13.342650 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5797e352-8645-4f03-b47a-77193f51f1a0 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:13.360778 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-ed01059e-b952-4ec9-9952-489c06a62e4e None None] MFA Rules not processed for user `332b1c9c68db4175a1383bb588b5e822`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:13.362013 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-ed01059e-b952-4ec9-9952-489c06a62e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:13.378754 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 826/1665] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:13.383284 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01059e-b952-4ec9-9952-489c06a62e4e None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:13.383284 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-ed01059e-b952-4ec9-9952-489c06a62e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:13.424451 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01059e-b952-4ec9-9952-489c06a62e4e None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:13.435463 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df5a32b-2328-48a1-af81-9dc6bc85e064 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:13.435950 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df5a32b-2328-48a1-af81-9dc6bc85e064 service nova] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:13.436308 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df5a32b-2328-48a1-af81-9dc6bc85e064 service nova] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:13.436489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df5a32b-2328-48a1-af81-9dc6bc85e064 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:13.456948 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df5a32b-2328-48a1-af81-9dc6bc85e064 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:13.458636 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df5a32b-2328-48a1-af81-9dc6bc85e064 service nova] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:13.495455 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 827/1666] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:33:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:33:13.496581 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df5a32b-2328-48a1-af81-9dc6bc85e064 service nova] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:13.502787 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-8df5a32b-2328-48a1-af81-9dc6bc85e064 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:13.532514 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df5a32b-2328-48a1-af81-9dc6bc85e064 service nova] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:13.542515 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020d6c7-5430-4829-8e86-f3f12f594e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:13.543672 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020d6c7-5430-4829-8e86-f3f12f594e6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:13.544029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020d6c7-5430-4829-8e86-f3f12f594e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:13.544427 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020d6c7-5430-4829-8e86-f3f12f594e6a admin admin] PATH_INFO: `/v3/users/0a5b43ce22cc424e87598501e12d256d` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:13.548859 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6020d6c7-5430-4829-8e86-f3f12f594e6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a5b43ce22cc424e87598501e12d256d)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:13.550770 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6020d6c7-5430-4829-8e86-f3f12f594e6a admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:13.565530 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-5797e352-8645-4f03-b47a-77193f51f1a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e2125ee287a40acbb9d30b8bfeadb08'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:13.566245 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-5797e352-8645-4f03-b47a-77193f51f1a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e2125ee287a40acbb9d30b8bfeadb08'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:13.588048 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-5797e352-8645-4f03-b47a-77193f51f1a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e2125ee287a40acbb9d30b8bfeadb08'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:13.588810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-5797e352-8645-4f03-b47a-77193f51f1a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e2125ee287a40acbb9d30b8bfeadb08'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:13.616063 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-5797e352-8645-4f03-b47a-77193f51f1a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e2125ee287a40acbb9d30b8bfeadb08'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:13.723967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 840/1667] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:33:13 2022] DELETE /identity/v3/users/6e2125ee287a40acbb9d30b8bfeadb08 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:13.725125 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5797e352-8645-4f03-b47a-77193f51f1a0 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:13.726655 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-5797e352-8645-4f03-b47a-77193f51f1a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:13.793377 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:13.798290 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-6020d6c7-5430-4829-8e86-f3f12f594e6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a5b43ce22cc424e87598501e12d256d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:13.823260 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-6020d6c7-5430-4829-8e86-f3f12f594e6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a5b43ce22cc424e87598501e12d256d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:13.836533 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5797e352-8645-4f03-b47a-77193f51f1a0 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:13.848566 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02afb6-39cc-44e5-81c5-13af611b5580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:13.849860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02afb6-39cc-44e5-81c5-13af611b5580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:13.850342 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02afb6-39cc-44e5-81c5-13af611b5580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:13.850772 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02afb6-39cc-44e5-81c5-13af611b5580 admin admin] PATH_INFO: `/v3/projects/d393195d103b48a1b8d17fc56b3ad594` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:13.852602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a02afb6-39cc-44e5-81c5-13af611b5580 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d393195d103b48a1b8d17fc56b3ad594)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:13.854877 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a02afb6-39cc-44e5-81c5-13af611b5580 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:13.863163 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-6020d6c7-5430-4829-8e86-f3f12f594e6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a5b43ce22cc424e87598501e12d256d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:13.864030 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-6020d6c7-5430-4829-8e86-f3f12f594e6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a5b43ce22cc424e87598501e12d256d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:13.960168 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-6020d6c7-5430-4829-8e86-f3f12f594e6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a5b43ce22cc424e87598501e12d256d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:13.970386 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 828/1668] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:33:13 2022] DELETE /identity/v3/users/0a5b43ce22cc424e87598501e12d256d => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:13.970386 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020d6c7-5430-4829-8e86-f3f12f594e6a admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:13.970386 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-6020d6c7-5430-4829-8e86-f3f12f594e6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:13.993063 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:14.061861 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-6a02afb6-39cc-44e5-81c5-13af611b5580 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd393195d103b48a1b8d17fc56b3ad594', 'request_id': 'req-6a02afb6-39cc-44e5-81c5-13af611b5580'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:14.084244 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-6a02afb6-39cc-44e5-81c5-13af611b5580 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd393195d103b48a1b8d17fc56b3ad594', 'request_id': 'req-6a02afb6-39cc-44e5-81c5-13af611b5580'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:14.100034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-6a02afb6-39cc-44e5-81c5-13af611b5580 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd393195d103b48a1b8d17fc56b3ad594', 'request_id': 'req-6a02afb6-39cc-44e5-81c5-13af611b5580'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:14.180992 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020d6c7-5430-4829-8e86-f3f12f594e6a admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:14.204524 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-6a02afb6-39cc-44e5-81c5-13af611b5580 admin admin] The token cache is being invalidate because project d393195d103b48a1b8d17fc56b3ad594 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:33:14.204816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-6a02afb6-39cc-44e5-81c5-13af611b5580 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:14.206563 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-6a02afb6-39cc-44e5-81c5-13af611b5580 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:14.212001 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 841/1669] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:33:13 2022] DELETE /identity/v3/projects/d393195d103b48a1b8d17fc56b3ad594 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:14.212001 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02afb6-39cc-44e5-81c5-13af611b5580 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:14.214033 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-6a02afb6-39cc-44e5-81c5-13af611b5580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:14.340496 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f722d30-3c24-4336-9fe7-8b0e3a621913 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:14.341350 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f722d30-3c24-4336-9fe7-8b0e3a621913 service nova] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:14.341669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f722d30-3c24-4336-9fe7-8b0e3a621913 service nova] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:14.341938 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f722d30-3c24-4336-9fe7-8b0e3a621913 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:14.343913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-7f722d30-3c24-4336-9fe7-8b0e3a621913 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:14.357725 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02afb6-39cc-44e5-81c5-13af611b5580 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:14.363701 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df5218-90ef-4f06-a065-22f8bbd0de33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:14.364644 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df5218-90ef-4f06-a065-22f8bbd0de33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:14.364944 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df5218-90ef-4f06-a065-22f8bbd0de33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:14.365246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df5218-90ef-4f06-a065-22f8bbd0de33 admin admin] PATH_INFO: `/v3/projects/6d85693835db4b36b8792c12ae7f76d0` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:14.366848 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df5218-90ef-4f06-a065-22f8bbd0de33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d85693835db4b36b8792c12ae7f76d0)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:14.369144 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df5218-90ef-4f06-a065-22f8bbd0de33 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:14.580545 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f722d30-3c24-4336-9fe7-8b0e3a621913 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:14.580545 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f722d30-3c24-4336-9fe7-8b0e3a621913 service nova] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:14.580545 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 829/1670] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:33:13 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:33:14.580545 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8c613-a74c-4f19-b38e-62baf990def6 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:14.580545 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8c613-a74c-4f19-b38e-62baf990def6 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:14.580545 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8c613-a74c-4f19-b38e-62baf990def6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:14.583205 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-acd8c613-a74c-4f19-b38e-62baf990def6 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:14.583205 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-acd8c613-a74c-4f19-b38e-62baf990def6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:14.583205 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 830/1671] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:14.583205 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a0131-91ba-47a1-8213-2e926c46bfe2 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:14.583205 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a0131-91ba-47a1-8213-2e926c46bfe2 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:14.583205 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a0131-91ba-47a1-8213-2e926c46bfe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:14.584964 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-8d8a0131-91ba-47a1-8213-2e926c46bfe2 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:14.584964 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-8d8a0131-91ba-47a1-8213-2e926c46bfe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:14.584964 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 831/1672] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:14.589326 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf3628-d6fc-4acb-bc80-7db6a6646511 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:14.591215 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf3628-d6fc-4acb-bc80-7db6a6646511 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:14.592435 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf3628-d6fc-4acb-bc80-7db6a6646511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:14.637149 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-60cf3628-d6fc-4acb-bc80-7db6a6646511 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:14.640743 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-60cf3628-d6fc-4acb-bc80-7db6a6646511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:14.668887 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 832/1673] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:14.670572 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cf3628-d6fc-4acb-bc80-7db6a6646511 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:14.674227 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-60cf3628-d6fc-4acb-bc80-7db6a6646511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:14.712810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-48df5218-90ef-4f06-a065-22f8bbd0de33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d85693835db4b36b8792c12ae7f76d0', 'request_id': 'req-48df5218-90ef-4f06-a065-22f8bbd0de33'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:14.725276 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-48df5218-90ef-4f06-a065-22f8bbd0de33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d85693835db4b36b8792c12ae7f76d0', 'request_id': 'req-48df5218-90ef-4f06-a065-22f8bbd0de33'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:14.745851 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-48df5218-90ef-4f06-a065-22f8bbd0de33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d85693835db4b36b8792c12ae7f76d0', 'request_id': 'req-48df5218-90ef-4f06-a065-22f8bbd0de33'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:14.804009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cf3628-d6fc-4acb-bc80-7db6a6646511 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:14.869793 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-48df5218-90ef-4f06-a065-22f8bbd0de33 admin admin] The token cache is being invalidate because project 6d85693835db4b36b8792c12ae7f76d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:33:14.870085 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-48df5218-90ef-4f06-a065-22f8bbd0de33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:14.871202 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-48df5218-90ef-4f06-a065-22f8bbd0de33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:14.873443 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 842/1674] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:33:14 2022] DELETE /identity/v3/projects/6d85693835db4b36b8792c12ae7f76d0 => generated 0 bytes in 664 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:14.874338 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df5218-90ef-4f06-a065-22f8bbd0de33 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:14.878295 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-48df5218-90ef-4f06-a065-22f8bbd0de33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:14.879254 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f295bcc-6580-4f80-8913-b5113041e651 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:14.880210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f295bcc-6580-4f80-8913-b5113041e651 service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:14.880733 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f295bcc-6580-4f80-8913-b5113041e651 service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:14.881227 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f295bcc-6580-4f80-8913-b5113041e651 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:14.883498 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-2f295bcc-6580-4f80-8913-b5113041e651 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:14.982170 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df5218-90ef-4f06-a065-22f8bbd0de33 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:14.988548 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f1ff82-5a85-4b3f-b9e9-c1fb14eae2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:14.990969 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1ff82-5a85-4b3f-b9e9-c1fb14eae2a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:14.990969 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1ff82-5a85-4b3f-b9e9-c1fb14eae2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:14.990969 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1ff82-5a85-4b3f-b9e9-c1fb14eae2a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:14.990969 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f1ff82-5a85-4b3f-b9e9-c1fb14eae2a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:33:14.991343 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f1ff82-5a85-4b3f-b9e9-c1fb14eae2a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:14.992806 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f1ff82-5a85-4b3f-b9e9-c1fb14eae2a4 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:15.008398 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 843/1675] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:33:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:33:15.009595 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f1ff82-5a85-4b3f-b9e9-c1fb14eae2a4 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:15.013263 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-19f1ff82-5a85-4b3f-b9e9-c1fb14eae2a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:15.021374 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f295bcc-6580-4f80-8913-b5113041e651 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:15.021374 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f295bcc-6580-4f80-8913-b5113041e651 service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:15.029133 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f1ff82-5a85-4b3f-b9e9-c1fb14eae2a4 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:15.035728 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35e558-f0b3-4aea-bd27-0b78bfcd8ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:15.036463 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35e558-f0b3-4aea-bd27-0b78bfcd8ba6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:15.036746 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35e558-f0b3-4aea-bd27-0b78bfcd8ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:15.037024 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35e558-f0b3-4aea-bd27-0b78bfcd8ba6 admin admin] PATH_INFO: `/v3/projects/d8e382508a4e4918ade240f2cefd9865` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:15.038546 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35e558-f0b3-4aea-bd27-0b78bfcd8ba6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8e382508a4e4918ade240f2cefd9865)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:15.040919 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35e558-f0b3-4aea-bd27-0b78bfcd8ba6 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:15.057743 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 833/1676] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:33:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:33:15.059286 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c84d7d6-48a3-4b6e-ab63-daa4403f7633 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:15.059802 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c84d7d6-48a3-4b6e-ab63-daa4403f7633 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:15.060178 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c84d7d6-48a3-4b6e-ab63-daa4403f7633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:15.089886 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-4c84d7d6-48a3-4b6e-ab63-daa4403f7633 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:15.091554 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-4c84d7d6-48a3-4b6e-ab63-daa4403f7633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:15.101560 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 834/1677] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:15.102493 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c84d7d6-48a3-4b6e-ab63-daa4403f7633 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:15.114816 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c84d7d6-48a3-4b6e-ab63-daa4403f7633 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:15.126817 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0184ddcd-53bf-4403-a604-d38678a0caf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:15.126817 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184ddcd-53bf-4403-a604-d38678a0caf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:15.126817 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184ddcd-53bf-4403-a604-d38678a0caf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:15.126817 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184ddcd-53bf-4403-a604-d38678a0caf4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:15.127228 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0184ddcd-53bf-4403-a604-d38678a0caf4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:15.127228 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0184ddcd-53bf-4403-a604-d38678a0caf4 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:15.191925 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-eb35e558-f0b3-4aea-bd27-0b78bfcd8ba6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8e382508a4e4918ade240f2cefd9865', 'request_id': 'req-eb35e558-f0b3-4aea-bd27-0b78bfcd8ba6'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:15.203665 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-eb35e558-f0b3-4aea-bd27-0b78bfcd8ba6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8e382508a4e4918ade240f2cefd9865', 'request_id': 'req-eb35e558-f0b3-4aea-bd27-0b78bfcd8ba6'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:15.208473 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-eb35e558-f0b3-4aea-bd27-0b78bfcd8ba6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8e382508a4e4918ade240f2cefd9865', 'request_id': 'req-eb35e558-f0b3-4aea-bd27-0b78bfcd8ba6'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:15.275230 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 835/1678] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:33:15 2022] POST /identity/v3/projects => generated 357 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:33:15.277336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1ac01-9a65-48d4-9fb1-5d5d1351f904 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:15.279327 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1ac01-9a65-48d4-9fb1-5d5d1351f904 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:15.279635 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1ac01-9a65-48d4-9fb1-5d5d1351f904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:15.305017 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-eb35e558-f0b3-4aea-bd27-0b78bfcd8ba6 admin admin] The token cache is being invalidate because project d8e382508a4e4918ade240f2cefd9865 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:33:15.307006 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-eb35e558-f0b3-4aea-bd27-0b78bfcd8ba6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:15.307006 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-eb35e558-f0b3-4aea-bd27-0b78bfcd8ba6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:15.310222 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 844/1679] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:33:15 2022] DELETE /identity/v3/projects/d8e382508a4e4918ade240f2cefd9865 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:15.316828 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35e558-f0b3-4aea-bd27-0b78bfcd8ba6 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:15.321505 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-eb35e558-f0b3-4aea-bd27-0b78bfcd8ba6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:15.349589 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-edf1ac01-9a65-48d4-9fb1-5d5d1351f904 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:15.350953 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-edf1ac01-9a65-48d4-9fb1-5d5d1351f904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:15.514328 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35e558-f0b3-4aea-bd27-0b78bfcd8ba6 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:15.550939 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 836/1680] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:15.555546 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171287f-fb91-4081-83d6-a09db68b4878 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:15.556029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171287f-fb91-4081-83d6-a09db68b4878 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:15.556319 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171287f-fb91-4081-83d6-a09db68b4878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:15.606359 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-0171287f-fb91-4081-83d6-a09db68b4878 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:15.607815 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-0171287f-fb91-4081-83d6-a09db68b4878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:15.634410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 837/1681] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:15.634410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0171287f-fb91-4081-83d6-a09db68b4878 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:15.659098 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0171287f-fb91-4081-83d6-a09db68b4878 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:15.671996 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c2f4c7-0776-4130-804e-05a0423ccd9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:15.673265 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2f4c7-0776-4130-804e-05a0423ccd9b admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:15.673590 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2f4c7-0776-4130-804e-05a0423ccd9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:15.673864 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2f4c7-0776-4130-804e-05a0423ccd9b admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:15.674087 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ce477-34fb-4e6e-be17-dd262c81d13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:15.674917 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ce477-34fb-4e6e-be17-dd262c81d13f admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:15.675372 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ce477-34fb-4e6e-be17-dd262c81d13f admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:15.675828 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ce477-34fb-4e6e-be17-dd262c81d13f admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:15.677382 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3ce477-34fb-4e6e-be17-dd262c81d13f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:33:15.678416 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3ce477-34fb-4e6e-be17-dd262c81d13f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:15.680978 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c2f4c7-0776-4130-804e-05a0423ccd9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:15.682934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c2f4c7-0776-4130-804e-05a0423ccd9b admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:15.683272 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3ce477-34fb-4e6e-be17-dd262c81d13f admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:15.722396 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 838/1682] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:33:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:33:15.728513 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21033b8-9797-4638-aac0-82f172920110 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:15.728513 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21033b8-9797-4638-aac0-82f172920110 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:15.728513 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21033b8-9797-4638-aac0-82f172920110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:15.774675 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-d21033b8-9797-4638-aac0-82f172920110 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:15.776292 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-d21033b8-9797-4638-aac0-82f172920110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:15.805120 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 839/1683] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:15.808031 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2e2cc-907b-434a-8067-2cb28acdbb63 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:15.809522 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2e2cc-907b-434a-8067-2cb28acdbb63 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:15.811554 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b2e2cc-907b-434a-8067-2cb28acdbb63 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:15.813659 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 840/1684] 10.222.0.25 () {54 vars in 912 bytes} [Thu Jul 7 15:33:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:33:15.816012 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b2e2cc-907b-434a-8067-2cb28acdbb63 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:15.859006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b2e2cc-907b-434a-8067-2cb28acdbb63 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:15.870650 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a5439-6309-4a04-9c49-355845044a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:15.871325 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a5439-6309-4a04-9c49-355845044a60 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:15.871630 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a5439-6309-4a04-9c49-355845044a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:15.871911 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a5439-6309-4a04-9c49-355845044a60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:15.872272 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 845/1685] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:33:15 2022] POST /identity/v3/users => generated 279 bytes in 561 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:33:15.873430 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c2f4c7-0776-4130-804e-05a0423ccd9b admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:15.875043 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a5439-6309-4a04-9c49-355845044a60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:15.880014 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a5439-6309-4a04-9c49-355845044a60 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:15.919212 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c2f4c7-0776-4130-804e-05a0423ccd9b admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:15.933089 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0de483-9326-41d4-a465-edd0d2ff92c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:15.933929 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0de483-9326-41d4-a465-edd0d2ff92c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:15.934302 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0de483-9326-41d4-a465-edd0d2ff92c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:15.934838 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0de483-9326-41d4-a465-edd0d2ff92c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:15.935798 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0de483-9326-41d4-a465-edd0d2ff92c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:33:15.936409 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0de483-9326-41d4-a465-edd0d2ff92c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:15.938571 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0de483-9326-41d4-a465-edd0d2ff92c8 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:15.973249 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 846/1686] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:33:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:33:15.977695 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38697f7-d6f5-4ae3-a9b4-4c532cac5ce0 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:15.978304 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38697f7-d6f5-4ae3-a9b4-4c532cac5ce0 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:15.978484 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38697f7-d6f5-4ae3-a9b4-4c532cac5ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:16.044110 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 841/1687] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:33:15 2022] POST /identity/v3/projects => generated 377 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:33:16.047692 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a5439-6309-4a04-9c49-355845044a60 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:16.126299 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-c38697f7-d6f5-4ae3-a9b4-4c532cac5ce0 None None] MFA Rules not processed for user `e79efaea155c46ecb9831956e68aec8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:16.130126 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-c38697f7-d6f5-4ae3-a9b4-4c532cac5ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:16.278750 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a5439-6309-4a04-9c49-355845044a60 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:16.285768 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897fca67-45d6-4d02-8356-9974d7cc74af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:16.285768 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fca67-45d6-4d02-8356-9974d7cc74af admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:16.285983 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fca67-45d6-4d02-8356-9974d7cc74af admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:16.286334 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fca67-45d6-4d02-8356-9974d7cc74af admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:16.286990 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897fca67-45d6-4d02-8356-9974d7cc74af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:16.295270 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897fca67-45d6-4d02-8356-9974d7cc74af admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:16.325043 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 842/1688] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:33:16.326061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897fca67-45d6-4d02-8356-9974d7cc74af admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:16.353948 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897fca67-45d6-4d02-8356-9974d7cc74af admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:16.363072 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 847/1689] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Jul 7 15:33:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:16.363072 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38697f7-d6f5-4ae3-a9b4-4c532cac5ce0 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:16.365875 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f8df25-d0c5-499a-86ae-5c865759dcfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:16.366560 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8df25-d0c5-499a-86ae-5c865759dcfd admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:16.366845 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8df25-d0c5-499a-86ae-5c865759dcfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:16.367123 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8df25-d0c5-499a-86ae-5c865759dcfd admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:16.368003 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f8df25-d0c5-499a-86ae-5c865759dcfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:16.378599 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f8df25-d0c5-499a-86ae-5c865759dcfd admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:16.385916 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:16.394325 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38697f7-d6f5-4ae3-a9b4-4c532cac5ce0 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:16.401365 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126af971-a16a-4111-8109-2d91a7635491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:16.420643 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126af971-a16a-4111-8109-2d91a7635491 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:16.420991 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126af971-a16a-4111-8109-2d91a7635491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:16.421282 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126af971-a16a-4111-8109-2d91a7635491 admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:16.422285 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126af971-a16a-4111-8109-2d91a7635491 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:16.424372 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126af971-a16a-4111-8109-2d91a7635491 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:16.537105 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 843/1690] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:33:16 2022] POST /identity/v3/projects => generated 365 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:33:16.537105 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7c312-6cd1-4215-8e7b-2002f912b71e None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:16.537105 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7c312-6cd1-4215-8e7b-2002f912b71e None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:16.537105 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7c312-6cd1-4215-8e7b-2002f912b71e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:16.543949 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 848/1691] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:33:16 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:33:16.554849 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126af971-a16a-4111-8109-2d91a7635491 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:16.587619 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126af971-a16a-4111-8109-2d91a7635491 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:16.614094 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-20a7c312-6cd1-4215-8e7b-2002f912b71e None None] MFA Rules not processed for user `b2e09539caff4d4698cf29c1ea22d365`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:16.615353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-20a7c312-6cd1-4215-8e7b-2002f912b71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:16.631932 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:16.760245 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2159d7-a3ff-4cd3-abed-ed415dfc430f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:16.768412 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2159d7-a3ff-4cd3-abed-ed415dfc430f admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:16.768412 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2159d7-a3ff-4cd3-abed-ed415dfc430f admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:16.769120 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 844/1692] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:16.769207 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2159d7-a3ff-4cd3-abed-ed415dfc430f admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:16.769282 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7c312-6cd1-4215-8e7b-2002f912b71e None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:16.769363 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2159d7-a3ff-4cd3-abed-ed415dfc430f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:16.769363 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2159d7-a3ff-4cd3-abed-ed415dfc430f admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:16.770666 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-20a7c312-6cd1-4215-8e7b-2002f912b71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:16.815405 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 849/1693] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:33:16.825193 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2159d7-a3ff-4cd3-abed-ed415dfc430f admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:16.851486 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2159d7-a3ff-4cd3-abed-ed415dfc430f admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:16.865660 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240825d8-e7fa-4440-84f4-b4256116afd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:16.866366 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240825d8-e7fa-4440-84f4-b4256116afd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:16.866653 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240825d8-e7fa-4440-84f4-b4256116afd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:16.866932 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240825d8-e7fa-4440-84f4-b4256116afd7 admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:16.868403 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240825d8-e7fa-4440-84f4-b4256116afd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:16.871679 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240825d8-e7fa-4440-84f4-b4256116afd7 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:16.927118 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7c312-6cd1-4215-8e7b-2002f912b71e None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:16.939489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca29c5-6c68-40dd-ab15-ff214684afe1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:16.940850 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca29c5-6c68-40dd-ab15-ff214684afe1 service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:16.943407 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca29c5-6c68-40dd-ab15-ff214684afe1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:16.943407 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca29c5-6c68-40dd-ab15-ff214684afe1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:16.970347 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 850/1694] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:33:16 2022] POST /identity/v3/users => generated 283 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:33:16.971213 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240825d8-e7fa-4440-84f4-b4256116afd7 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:16.981235 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ca29c5-6c68-40dd-ab15-ff214684afe1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:16.990492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ca29c5-6c68-40dd-ab15-ff214684afe1 service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:16.991070 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240825d8-e7fa-4440-84f4-b4256116afd7 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:17.005692 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e62259-0b89-4484-b12c-29ceb58f051d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:17.006441 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e62259-0b89-4484-b12c-29ceb58f051d admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:17.006734 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e62259-0b89-4484-b12c-29ceb58f051d admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:17.007165 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e62259-0b89-4484-b12c-29ceb58f051d admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:17.007947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e62259-0b89-4484-b12c-29ceb58f051d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:17.008441 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 845/1695] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:33:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:33:17.010068 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca29c5-6c68-40dd-ab15-ff214684afe1 service neutron] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:17.010263 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e62259-0b89-4484-b12c-29ceb58f051d admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:17.011868 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-f7ca29c5-6c68-40dd-ab15-ff214684afe1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:17.035442 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 851/1696] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:33:17.036415 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e62259-0b89-4484-b12c-29ceb58f051d admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:17.043778 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-e0e62259-0b89-4484-b12c-29ceb58f051d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:17.069425 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e62259-0b89-4484-b12c-29ceb58f051d admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:17.083691 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f6e766-6bfa-4dec-860a-a6c280310e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:17.084646 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f6e766-6bfa-4dec-860a-a6c280310e51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:17.084936 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f6e766-6bfa-4dec-860a-a6c280310e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:17.085213 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f6e766-6bfa-4dec-860a-a6c280310e51 admin admin] PATH_INFO: `/v3/projects/4aa164cff08d4e7f8206ee70c01e08ed` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:17.086511 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f6e766-6bfa-4dec-860a-a6c280310e51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4aa164cff08d4e7f8206ee70c01e08ed)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:17.088653 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f6e766-6bfa-4dec-860a-a6c280310e51 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:17.258473 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca29c5-6c68-40dd-ab15-ff214684afe1 service neutron] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:17.288701 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-03f6e766-6bfa-4dec-860a-a6c280310e51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4aa164cff08d4e7f8206ee70c01e08ed', 'request_id': 'req-03f6e766-6bfa-4dec-860a-a6c280310e51'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:17.301169 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-03f6e766-6bfa-4dec-860a-a6c280310e51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4aa164cff08d4e7f8206ee70c01e08ed', 'request_id': 'req-03f6e766-6bfa-4dec-860a-a6c280310e51'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:17.314043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-03f6e766-6bfa-4dec-860a-a6c280310e51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4aa164cff08d4e7f8206ee70c01e08ed', 'request_id': 'req-03f6e766-6bfa-4dec-860a-a6c280310e51'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:17.414862 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b551c40-b545-421f-9cd5-ddcb807a04f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:17.415303 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b551c40-b545-421f-9cd5-ddcb807a04f3 service nova] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:17.415515 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b551c40-b545-421f-9cd5-ddcb807a04f3 service nova] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:17.415796 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b551c40-b545-421f-9cd5-ddcb807a04f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:17.441242 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-03f6e766-6bfa-4dec-860a-a6c280310e51 admin admin] The token cache is being invalidate because project 4aa164cff08d4e7f8206ee70c01e08ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:33:17.441703 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-03f6e766-6bfa-4dec-860a-a6c280310e51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:17.442585 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-03f6e766-6bfa-4dec-860a-a6c280310e51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:17.454454 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 852/1697] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:33:17 2022] DELETE /identity/v3/projects/4aa164cff08d4e7f8206ee70c01e08ed => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:17.454454 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f6e766-6bfa-4dec-860a-a6c280310e51 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:17.455170 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-03f6e766-6bfa-4dec-860a-a6c280310e51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:17.582045 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b551c40-b545-421f-9cd5-ddcb807a04f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:17.583720 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b551c40-b545-421f-9cd5-ddcb807a04f3 service nova] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:17.609709 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 846/1698] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:33:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 593 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:33:17.612946 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b551c40-b545-421f-9cd5-ddcb807a04f3 service nova] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:17.666893 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f6e766-6bfa-4dec-860a-a6c280310e51 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:17.684794 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f46c23-1d6c-478f-8666-5bb44dbd0bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:17.684794 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f46c23-1d6c-478f-8666-5bb44dbd0bdb admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:17.684794 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f46c23-1d6c-478f-8666-5bb44dbd0bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:17.684794 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f46c23-1d6c-478f-8666-5bb44dbd0bdb admin admin] PATH_INFO: `/v3/projects/12a7c17b37ab40999cf11094f358f8ed/users/1fb7b53dac284a62b694e0e6af0a7a89/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:17.715217 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f46c23-1d6c-478f-8666-5bb44dbd0bdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=12a7c17b37ab40999cf11094f358f8ed, user_id=1fb7b53dac284a62b694e0e6af0a7a89, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:17.719484 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f46c23-1d6c-478f-8666-5bb44dbd0bdb admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:17.785981 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b551c40-b545-421f-9cd5-ddcb807a04f3 service nova] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:17.794982 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cf5465-d2be-4274-94ce-5f7d972c953e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:17.797493 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf5465-d2be-4274-94ce-5f7d972c953e admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:17.797804 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf5465-d2be-4274-94ce-5f7d972c953e admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:17.798089 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf5465-d2be-4274-94ce-5f7d972c953e admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:17.798852 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cf5465-d2be-4274-94ce-5f7d972c953e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:17.800522 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cf5465-d2be-4274-94ce-5f7d972c953e admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:17.845494 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 853/1699] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:33:17 2022] PUT /identity/v3/projects/12a7c17b37ab40999cf11094f358f8ed/users/1fb7b53dac284a62b694e0e6af0a7a89/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:17.846550 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f46c23-1d6c-478f-8666-5bb44dbd0bdb admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:17.855503 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-66f46c23-1d6c-478f-8666-5bb44dbd0bdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:17.862703 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 847/1700] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:33:17.867606 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04b1ca-8a87-4071-8a42-5f87b7ff84ee None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:17.867606 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04b1ca-8a87-4071-8a42-5f87b7ff84ee None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:17.867606 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04b1ca-8a87-4071-8a42-5f87b7ff84ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:17.928218 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-ad04b1ca-8a87-4071-8a42-5f87b7ff84ee None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:17.930627 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-ad04b1ca-8a87-4071-8a42-5f87b7ff84ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:18.068818 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f46c23-1d6c-478f-8666-5bb44dbd0bdb admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:18.088442 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb6ea4-c57d-4785-9944-2d6434380a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:18.091670 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb6ea4-c57d-4785-9944-2d6434380a04 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:18.112569 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb6ea4-c57d-4785-9944-2d6434380a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:18.120515 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb6ea4-c57d-4785-9944-2d6434380a04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:18.120515 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafb6ea4-c57d-4785-9944-2d6434380a04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:18.121875 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafb6ea4-c57d-4785-9944-2d6434380a04 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:18.142327 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 848/1701] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:18.159175 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36691058-2a18-40f9-a1c8-169afd496ddf None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:18.159820 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36691058-2a18-40f9-a1c8-169afd496ddf None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:18.160804 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36691058-2a18-40f9-a1c8-169afd496ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:18.166189 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 854/1702] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:33:18.167148 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb6ea4-c57d-4785-9944-2d6434380a04 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:18.171225 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-bafb6ea4-c57d-4785-9944-2d6434380a04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:18.205034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb6ea4-c57d-4785-9944-2d6434380a04 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:18.231499 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e1122-1c71-449b-a561-29e57dc8e264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:18.235012 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e1122-1c71-449b-a561-29e57dc8e264 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:18.235343 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e1122-1c71-449b-a561-29e57dc8e264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:18.235687 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e1122-1c71-449b-a561-29e57dc8e264 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:18.236500 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e1122-1c71-449b-a561-29e57dc8e264 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:18.246778 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e1122-1c71-449b-a561-29e57dc8e264 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:18.248496 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-36691058-2a18-40f9-a1c8-169afd496ddf None None] MFA Rules not processed for user `1fb7b53dac284a62b694e0e6af0a7a89`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:18.253151 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-36691058-2a18-40f9-a1c8-169afd496ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:18.286709 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 855/1703] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:33:18.291243 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e0940a-0ebb-457b-80f3-1c599268a323 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:18.291544 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e0940a-0ebb-457b-80f3-1c599268a323 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:18.291952 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e0940a-0ebb-457b-80f3-1c599268a323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:18.348619 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-d9e0940a-0ebb-457b-80f3-1c599268a323 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:18.352845 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-d9e0940a-0ebb-457b-80f3-1c599268a323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:18.389584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 856/1704] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:18.390437 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e0940a-0ebb-457b-80f3-1c599268a323 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:18.417104 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e0940a-0ebb-457b-80f3-1c599268a323 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:18.423392 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da6731f-6d35-4950-b920-10ecccacdb47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:18.424450 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6731f-6d35-4950-b920-10ecccacdb47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:18.424729 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6731f-6d35-4950-b920-10ecccacdb47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:18.425042 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6731f-6d35-4950-b920-10ecccacdb47 admin admin] PATH_INFO: `/v3/projects/82288fd065744001b46a7b4cadc9b26c/users/8670c0d680a64e1684e8be44fe005724/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:18.450118 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 849/1705] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:18 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:18.450118 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36691058-2a18-40f9-a1c8-169afd496ddf None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:18.461586 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da6731f-6d35-4950-b920-10ecccacdb47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82288fd065744001b46a7b4cadc9b26c, user_id=8670c0d680a64e1684e8be44fe005724, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:18.462995 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36691058-2a18-40f9-a1c8-169afd496ddf None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:18.468407 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da6731f-6d35-4950-b920-10ecccacdb47 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:18.471027 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb83e18-5b70-42d6-90be-96b72403f26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:18.471967 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb83e18-5b70-42d6-90be-96b72403f26d admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:18.480620 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb83e18-5b70-42d6-90be-96b72403f26d admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:18.480620 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb83e18-5b70-42d6-90be-96b72403f26d admin admin] PATH_INFO: `/v3/projects/6c773c85cddf49a99b435c9534205e44/users/7b468cfe72fb427383fd161ea1826b4d/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:18.494175 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb83e18-5b70-42d6-90be-96b72403f26d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c773c85cddf49a99b435c9534205e44, user_id=7b468cfe72fb427383fd161ea1826b4d, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:18.497188 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb83e18-5b70-42d6-90be-96b72403f26d admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:18.582756 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:18.635864 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 857/1706] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:33:18 2022] PUT /identity/v3/projects/82288fd065744001b46a7b4cadc9b26c/users/8670c0d680a64e1684e8be44fe005724/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:18.636973 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da6731f-6d35-4950-b920-10ecccacdb47 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:18.648772 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-4da6731f-6d35-4950-b920-10ecccacdb47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:18.650953 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 850/1707] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:33:18 2022] PUT /identity/v3/projects/6c773c85cddf49a99b435c9534205e44/users/7b468cfe72fb427383fd161ea1826b4d/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:18.652492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb83e18-5b70-42d6-90be-96b72403f26d admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:18.805301 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:18.813364 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da6731f-6d35-4950-b920-10ecccacdb47 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:18.824198 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da604e9-44b8-438c-b989-fd618cc9a488 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '755c2782cfb042f89950b8fc862e4fa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:18.825794 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da604e9-44b8-438c-b989-fd618cc9a488 service placement] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:18.826296 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da604e9-44b8-438c-b989-fd618cc9a488 service placement] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:18.826923 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da604e9-44b8-438c-b989-fd618cc9a488 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:18.830971 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-2da604e9-44b8-438c-b989-fd618cc9a488 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:18.859056 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb83e18-5b70-42d6-90be-96b72403f26d admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:18.876189 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da604e9-44b8-438c-b989-fd618cc9a488 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:18.877824 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da604e9-44b8-438c-b989-fd618cc9a488 service placement] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:18.901088 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 858/1708] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:33:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:33:18.902080 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da604e9-44b8-438c-b989-fd618cc9a488 service placement] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:19.000108 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da604e9-44b8-438c-b989-fd618cc9a488 service placement] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:19.009733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57bb08d-3cfc-4d7a-b38d-54f94257d482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:19.009733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57bb08d-3cfc-4d7a-b38d-54f94257d482 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:19.009733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57bb08d-3cfc-4d7a-b38d-54f94257d482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:19.009733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57bb08d-3cfc-4d7a-b38d-54f94257d482 admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:19.010457 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57bb08d-3cfc-4d7a-b38d-54f94257d482 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:19.010457 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57bb08d-3cfc-4d7a-b38d-54f94257d482 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:19.029251 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7321922-7365-48bc-99a3-96a87943ef4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:19.030025 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7321922-7365-48bc-99a3-96a87943ef4a admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:19.030394 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7321922-7365-48bc-99a3-96a87943ef4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:19.030678 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7321922-7365-48bc-99a3-96a87943ef4a admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:19.032501 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7321922-7365-48bc-99a3-96a87943ef4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:33:19.034358 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7321922-7365-48bc-99a3-96a87943ef4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:19.035125 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7321922-7365-48bc-99a3-96a87943ef4a admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:19.050283 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 851/1709] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:33:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:33:19.052591 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0d489-7719-4a1a-95f3-4543355b9042 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:19.052880 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0d489-7719-4a1a-95f3-4543355b9042 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:19.053237 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0d489-7719-4a1a-95f3-4543355b9042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:19.126001 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-49c0d489-7719-4a1a-95f3-4543355b9042 None None] MFA Rules not processed for user `8670c0d680a64e1684e8be44fe005724`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:19.126001 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-49c0d489-7719-4a1a-95f3-4543355b9042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:19.126161 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 859/1710] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:33:18 2022] POST /identity/v3/projects => generated 355 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:33:19.130396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6b8e7-307d-4fb1-b42b-abfde6a73a54 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:19.132078 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6b8e7-307d-4fb1-b42b-abfde6a73a54 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:19.132426 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6b8e7-307d-4fb1-b42b-abfde6a73a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:19.211297 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-91c6b8e7-307d-4fb1-b42b-abfde6a73a54 None None] MFA Rules not processed for user `7b468cfe72fb427383fd161ea1826b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:19.212573 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-91c6b8e7-307d-4fb1-b42b-abfde6a73a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:19.220170 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 852/1711] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:19.222241 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c0d489-7719-4a1a-95f3-4543355b9042 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:19.236323 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c0d489-7719-4a1a-95f3-4543355b9042 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:19.330880 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 860/1712] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:19 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:19.330880 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6b8e7-307d-4fb1-b42b-abfde6a73a54 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:19.379168 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e7287-11ee-49e6-a7de-b66125d22011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:19.379764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e7287-11ee-49e6-a7de-b66125d22011 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:19.380049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e7287-11ee-49e6-a7de-b66125d22011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:19.380693 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e7287-11ee-49e6-a7de-b66125d22011 admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:19.382528 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781e7287-11ee-49e6-a7de-b66125d22011 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:19.384774 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781e7287-11ee-49e6-a7de-b66125d22011 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:19.448131 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6b8e7-307d-4fb1-b42b-abfde6a73a54 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:19.454792 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0beb183-f2b3-4728-97f1-e06a6aefbebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:19.455817 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0beb183-f2b3-4728-97f1-e06a6aefbebb admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:19.456410 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0beb183-f2b3-4728-97f1-e06a6aefbebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:19.456874 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0beb183-f2b3-4728-97f1-e06a6aefbebb admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:19.457754 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0beb183-f2b3-4728-97f1-e06a6aefbebb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:19.460022 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0beb183-f2b3-4728-97f1-e06a6aefbebb admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:19.509106 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 853/1713] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:33:19 2022] POST /identity/v3/projects => generated 405 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:33:19.511729 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039d85e-99b7-497d-8c3c-9d103e9dc869 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:19.512013 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039d85e-99b7-497d-8c3c-9d103e9dc869 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:19.512293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039d85e-99b7-497d-8c3c-9d103e9dc869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:19.535915 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-5039d85e-99b7-497d-8c3c-9d103e9dc869 None None] MFA Rules not processed for user `b5dec80a83aa463b99bd298ab09dfbdb`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:19.537218 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-5039d85e-99b7-497d-8c3c-9d103e9dc869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:19.594859 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 861/1714] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:33:19 2022] POST /identity/v3/users => generated 284 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:33:19.596863 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0beb183-f2b3-4728-97f1-e06a6aefbebb admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:19.622675 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0beb183-f2b3-4728-97f1-e06a6aefbebb admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:19.670628 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 854/1715] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:19.672846 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5039d85e-99b7-497d-8c3c-9d103e9dc869 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:19.768444 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544dce2-870c-450e-a5ed-3c0ead14fb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:19.769558 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544dce2-870c-450e-a5ed-3c0ead14fb78 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:19.770030 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544dce2-870c-450e-a5ed-3c0ead14fb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:19.770669 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544dce2-870c-450e-a5ed-3c0ead14fb78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:19.771875 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b544dce2-870c-450e-a5ed-3c0ead14fb78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:19.772101 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5039d85e-99b7-497d-8c3c-9d103e9dc869 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:19.774691 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b544dce2-870c-450e-a5ed-3c0ead14fb78 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:19.777541 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155956ac-6edf-4f8a-8b7d-27358e7801ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:19.778507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155956ac-6edf-4f8a-8b7d-27358e7801ca admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:19.778967 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155956ac-6edf-4f8a-8b7d-27358e7801ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:19.779359 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155956ac-6edf-4f8a-8b7d-27358e7801ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:19.780181 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155956ac-6edf-4f8a-8b7d-27358e7801ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:19.782725 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155956ac-6edf-4f8a-8b7d-27358e7801ca admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:19.827416 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:19.895849 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 862/1716] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:33:19 2022] POST /identity/v3/projects => generated 375 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:33:19.897201 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544dce2-870c-450e-a5ed-3c0ead14fb78 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:19.937561 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 855/1717] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:33:19 2022] POST /identity/v3/projects => generated 365 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:33:19.938642 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155956ac-6edf-4f8a-8b7d-27358e7801ca admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:20.133957 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544dce2-870c-450e-a5ed-3c0ead14fb78 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:20.155978 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155956ac-6edf-4f8a-8b7d-27358e7801ca admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:20.162342 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87716f5c-0e82-4841-b08d-d0fb200f6197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:20.163216 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87716f5c-0e82-4841-b08d-d0fb200f6197 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:20.163716 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87716f5c-0e82-4841-b08d-d0fb200f6197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:20.164150 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87716f5c-0e82-4841-b08d-d0fb200f6197 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:20.165055 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87716f5c-0e82-4841-b08d-d0fb200f6197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:20.167813 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87716f5c-0e82-4841-b08d-d0fb200f6197 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:20.182121 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 856/1718] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:33:20.183368 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87716f5c-0e82-4841-b08d-d0fb200f6197 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:20.195854 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87716f5c-0e82-4841-b08d-d0fb200f6197 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:20.201979 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987789ea-ef20-4181-963b-1866a85e2bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:20.202974 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987789ea-ef20-4181-963b-1866a85e2bb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:20.204609 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987789ea-ef20-4181-963b-1866a85e2bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:20.205104 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987789ea-ef20-4181-963b-1866a85e2bb1 admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:20.205995 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987789ea-ef20-4181-963b-1866a85e2bb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:20.208792 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987789ea-ef20-4181-963b-1866a85e2bb1 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:20.283184 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aadd7e-1312-4ba5-8513-e8a7b4aeb47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:20.283913 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aadd7e-1312-4ba5-8513-e8a7b4aeb47c admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:20.284342 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aadd7e-1312-4ba5-8513-e8a7b4aeb47c admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:20.284453 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aadd7e-1312-4ba5-8513-e8a7b4aeb47c admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:20.285169 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aadd7e-1312-4ba5-8513-e8a7b4aeb47c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:20.287115 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aadd7e-1312-4ba5-8513-e8a7b4aeb47c admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:20.326199 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 857/1719] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:33:20 2022] POST /identity/v3/users => generated 294 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:33:20.327020 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987789ea-ef20-4181-963b-1866a85e2bb1 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:20.341099 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987789ea-ef20-4181-963b-1866a85e2bb1 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:20.350511 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637cd6e-68a9-46e5-ae05-43bb2e51d329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:20.351649 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637cd6e-68a9-46e5-ae05-43bb2e51d329 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:20.358951 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637cd6e-68a9-46e5-ae05-43bb2e51d329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:20.358951 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637cd6e-68a9-46e5-ae05-43bb2e51d329 admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:20.358951 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3637cd6e-68a9-46e5-ae05-43bb2e51d329 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:20.358951 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3637cd6e-68a9-46e5-ae05-43bb2e51d329 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:20.442793 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 863/1720] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:33:19 2022] POST /identity/v3/users => generated 303 bytes in 546 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:33:20.444426 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e04211-fb71-436e-98e8-552c391fb350 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:20.444728 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e04211-fb71-436e-98e8-552c391fb350 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:20.445008 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e04211-fb71-436e-98e8-552c391fb350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:20.464308 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-f8e04211-fb71-436e-98e8-552c391fb350 None None] MFA Rules not processed for user `2033ad40d45747aa8294e4feb5a6cfe0`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:20.465541 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-f8e04211-fb71-436e-98e8-552c391fb350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:20.487717 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 858/1721] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:33:20 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:33:20.491015 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637cd6e-68a9-46e5-ae05-43bb2e51d329 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:20.499198 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637cd6e-68a9-46e5-ae05-43bb2e51d329 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:20.504667 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6950dea-4b9b-4dfd-a98c-ac3e7158d1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:20.505738 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6950dea-4b9b-4dfd-a98c-ac3e7158d1d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:20.506178 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6950dea-4b9b-4dfd-a98c-ac3e7158d1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:20.506595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6950dea-4b9b-4dfd-a98c-ac3e7158d1d4 admin admin] PATH_INFO: `/v3/projects/cc8188fbcf844769b24baec11728941c/users/7cd9c89376814399ab81b7ad849f8484/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:20.522998 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6950dea-4b9b-4dfd-a98c-ac3e7158d1d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc8188fbcf844769b24baec11728941c, user_id=7cd9c89376814399ab81b7ad849f8484, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:20.529060 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 864/1722] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:20.529060 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e04211-fb71-436e-98e8-552c391fb350 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:20.532694 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6950dea-4b9b-4dfd-a98c-ac3e7158d1d4 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:20.549720 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e04211-fb71-436e-98e8-552c391fb350 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:20.556353 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bca590-dcb7-4e36-95cf-49561fa15635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:20.557179 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bca590-dcb7-4e36-95cf-49561fa15635 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:20.557629 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bca590-dcb7-4e36-95cf-49561fa15635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:20.557972 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bca590-dcb7-4e36-95cf-49561fa15635 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:20.558775 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bca590-dcb7-4e36-95cf-49561fa15635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:20.560496 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bca590-dcb7-4e36-95cf-49561fa15635 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:20.582166 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 865/1723] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:33:20.583712 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21536b-9808-4c3f-880b-2cad95e6e097 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:20.583983 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21536b-9808-4c3f-880b-2cad95e6e097 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:20.584260 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21536b-9808-4c3f-880b-2cad95e6e097 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:20.586520 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 866/1724] 10.222.0.25 () {54 vars in 912 bytes} [Thu Jul 7 15:33:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:33:20.587226 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21536b-9808-4c3f-880b-2cad95e6e097 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:20.645540 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 859/1725] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:33:20 2022] PUT /identity/v3/projects/cc8188fbcf844769b24baec11728941c/users/7cd9c89376814399ab81b7ad849f8484/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:20.645540 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6950dea-4b9b-4dfd-a98c-ac3e7158d1d4 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:20.661927 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21536b-9808-4c3f-880b-2cad95e6e097 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:20.792154 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf20037c-f436-407a-b877-3db6672ab939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:20.792712 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6950dea-4b9b-4dfd-a98c-ac3e7158d1d4 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:20.793298 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf20037c-f436-407a-b877-3db6672ab939 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:20.793734 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf20037c-f436-407a-b877-3db6672ab939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:20.794192 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf20037c-f436-407a-b877-3db6672ab939 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:20.795130 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf20037c-f436-407a-b877-3db6672ab939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:20.797117 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf20037c-f436-407a-b877-3db6672ab939 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:20.798822 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d680ba1-09a2-4679-aad3-51f3ea5a8cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:20.799483 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d680ba1-09a2-4679-aad3-51f3ea5a8cec admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:20.799891 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d680ba1-09a2-4679-aad3-51f3ea5a8cec admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:20.800517 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d680ba1-09a2-4679-aad3-51f3ea5a8cec admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:20.801089 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d680ba1-09a2-4679-aad3-51f3ea5a8cec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:20.802773 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d680ba1-09a2-4679-aad3-51f3ea5a8cec admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:20.810253 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 867/1726] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:33:20.811479 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf20037c-f436-407a-b877-3db6672ab939 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:20.824659 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf20037c-f436-407a-b877-3db6672ab939 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:20.827984 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 860/1727] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:33:20.827984 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8906459-d1c0-4c96-8408-9c7f0905120b None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:20.827984 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8906459-d1c0-4c96-8408-9c7f0905120b None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:20.827984 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8906459-d1c0-4c96-8408-9c7f0905120b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:20.834804 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6756f50e-01a1-46d8-a7f9-37e680a5d399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:20.836202 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6756f50e-01a1-46d8-a7f9-37e680a5d399 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:20.836680 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6756f50e-01a1-46d8-a7f9-37e680a5d399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:20.837139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6756f50e-01a1-46d8-a7f9-37e680a5d399 admin admin] PATH_INFO: `/v3/projects/819cabbf14ab4efab66962d1621ad00c/users/2ef3a193faab46f68e419eece886fd23/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:20.857753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6756f50e-01a1-46d8-a7f9-37e680a5d399 admin admin] RBAC: Authorizing `identity:create_grant(project_id=819cabbf14ab4efab66962d1621ad00c, user_id=2ef3a193faab46f68e419eece886fd23, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:20.861314 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-c8906459-d1c0-4c96-8408-9c7f0905120b None None] MFA Rules not processed for user `e79efaea155c46ecb9831956e68aec8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:20.861988 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6756f50e-01a1-46d8-a7f9-37e680a5d399 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:20.862699 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-c8906459-d1c0-4c96-8408-9c7f0905120b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:20.955467 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 861/1728] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Jul 7 15:33:20 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:20.963377 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8906459-d1c0-4c96-8408-9c7f0905120b None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:20.973027 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 868/1729] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:33:20 2022] PUT /identity/v3/projects/819cabbf14ab4efab66962d1621ad00c/users/2ef3a193faab46f68e419eece886fd23/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:20.975894 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6756f50e-01a1-46d8-a7f9-37e680a5d399 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:21.009139 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8906459-d1c0-4c96-8408-9c7f0905120b None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:21.108312 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdb7d0e-ee60-4b42-8278-06ba6b7cfd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:21.108825 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6756f50e-01a1-46d8-a7f9-37e680a5d399 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:21.109033 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb7d0e-ee60-4b42-8278-06ba6b7cfd23 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:21.110231 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb7d0e-ee60-4b42-8278-06ba6b7cfd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:21.110346 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb7d0e-ee60-4b42-8278-06ba6b7cfd23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:21.111078 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdb7d0e-ee60-4b42-8278-06ba6b7cfd23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:21.113552 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdb7d0e-ee60-4b42-8278-06ba6b7cfd23 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:21.120781 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5596fdb7-a776-478a-bb9f-b0cc09e5eedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:21.121883 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596fdb7-a776-478a-bb9f-b0cc09e5eedb admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:21.122508 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596fdb7-a776-478a-bb9f-b0cc09e5eedb admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:21.123030 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596fdb7-a776-478a-bb9f-b0cc09e5eedb admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:21.124093 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5596fdb7-a776-478a-bb9f-b0cc09e5eedb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:21.127733 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 862/1730] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:33:21.128139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5596fdb7-a776-478a-bb9f-b0cc09e5eedb admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:21.128791 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdb7d0e-ee60-4b42-8278-06ba6b7cfd23 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:21.143249 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdb7d0e-ee60-4b42-8278-06ba6b7cfd23 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:21.144926 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 869/1731] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:33:21.146791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5596fdb7-a776-478a-bb9f-b0cc09e5eedb admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:21.152051 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf588a-4a29-40ed-9c32-f8c0fc7ce2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:21.152962 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf588a-4a29-40ed-9c32-f8c0fc7ce2c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:21.153415 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf588a-4a29-40ed-9c32-f8c0fc7ce2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:21.153855 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf588a-4a29-40ed-9c32-f8c0fc7ce2c6 admin admin] PATH_INFO: `/v3/projects/454dc6a84056496b9e26c6adf9000cfc/users/9fe991dc33f3440498f895fca6ba8f1c/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:21.160540 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5596fdb7-a776-478a-bb9f-b0cc09e5eedb admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:21.166571 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c381c-aaf9-4baa-937b-47994c903f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:21.170287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c381c-aaf9-4baa-937b-47994c903f11 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:21.170287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c381c-aaf9-4baa-937b-47994c903f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:21.170287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c381c-aaf9-4baa-937b-47994c903f11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:21.170287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c381c-aaf9-4baa-937b-47994c903f11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:21.171493 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c381c-aaf9-4baa-937b-47994c903f11 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:21.175816 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf588a-4a29-40ed-9c32-f8c0fc7ce2c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=454dc6a84056496b9e26c6adf9000cfc, user_id=9fe991dc33f3440498f895fca6ba8f1c, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:21.178898 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf588a-4a29-40ed-9c32-f8c0fc7ce2c6 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:21.186728 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 870/1732] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:33:21.188707 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c381c-aaf9-4baa-937b-47994c903f11 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:21.191493 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-f25c381c-aaf9-4baa-937b-47994c903f11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:21.294315 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c381c-aaf9-4baa-937b-47994c903f11 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:21.307369 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7acca01-f7d0-4981-b1b9-32553d7c6296 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:21.307369 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7acca01-f7d0-4981-b1b9-32553d7c6296 service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:21.307369 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7acca01-f7d0-4981-b1b9-32553d7c6296 service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:21.307369 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7acca01-f7d0-4981-b1b9-32553d7c6296 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:21.307753 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 863/1733] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:33:21 2022] PUT /identity/v3/projects/454dc6a84056496b9e26c6adf9000cfc/users/9fe991dc33f3440498f895fca6ba8f1c/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:21.307753 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf588a-4a29-40ed-9c32-f8c0fc7ce2c6 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:21.316360 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7acca01-f7d0-4981-b1b9-32553d7c6296 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:21.317805 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7acca01-f7d0-4981-b1b9-32553d7c6296 service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:21.336453 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 871/1734] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:33:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:33:21.337339 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7acca01-f7d0-4981-b1b9-32553d7c6296 service neutron] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:21.360413 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7acca01-f7d0-4981-b1b9-32553d7c6296 service neutron] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:21.398691 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:21.476913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf588a-4a29-40ed-9c32-f8c0fc7ce2c6 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:21.489829 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e81025c-3cdf-45e7-bfe2-9e27eeaf40a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:21.490829 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81025c-3cdf-45e7-bfe2-9e27eeaf40a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:21.491115 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81025c-3cdf-45e7-bfe2-9e27eeaf40a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:21.491426 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81025c-3cdf-45e7-bfe2-9e27eeaf40a9 admin admin] PATH_INFO: `/v3/projects/4fff94c31fa44e69834b368832ab9a45/users/e5d4739daa8b46618fff36718c462d17/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:21.492971 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a373db5-4c85-4595-9e8f-fdca29d27f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:21.493984 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a373db5-4c85-4595-9e8f-fdca29d27f89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:21.494372 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a373db5-4c85-4595-9e8f-fdca29d27f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:21.495119 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a373db5-4c85-4595-9e8f-fdca29d27f89 admin admin] PATH_INFO: `/v3/projects/cc8188fbcf844769b24baec11728941c/users/7cd9c89376814399ab81b7ad849f8484/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:21.498643 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a373db5-4c85-4595-9e8f-fdca29d27f89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc8188fbcf844769b24baec11728941c, user_id=7cd9c89376814399ab81b7ad849f8484, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:21.502012 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a373db5-4c85-4595-9e8f-fdca29d27f89 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:21.517084 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e81025c-3cdf-45e7-bfe2-9e27eeaf40a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fff94c31fa44e69834b368832ab9a45, user_id=e5d4739daa8b46618fff36718c462d17, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:21.522606 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 864/1735] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:33:21 2022] PUT /identity/v3/projects/cc8188fbcf844769b24baec11728941c/users/7cd9c89376814399ab81b7ad849f8484/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:21.522804 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e81025c-3cdf-45e7-bfe2-9e27eeaf40a9 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:21.522912 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a373db5-4c85-4595-9e8f-fdca29d27f89 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:21.596109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a373db5-4c85-4595-9e8f-fdca29d27f89 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:21.603350 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f579505-ddd8-447b-b05e-69358fcd84db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:21.604297 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f579505-ddd8-447b-b05e-69358fcd84db admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:21.604579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f579505-ddd8-447b-b05e-69358fcd84db admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:21.604851 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f579505-ddd8-447b-b05e-69358fcd84db admin admin] PATH_INFO: `/v3/projects/819cabbf14ab4efab66962d1621ad00c/users/2ef3a193faab46f68e419eece886fd23/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:21.607095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f579505-ddd8-447b-b05e-69358fcd84db admin admin] RBAC: Authorizing `identity:create_grant(project_id=819cabbf14ab4efab66962d1621ad00c, user_id=2ef3a193faab46f68e419eece886fd23, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:21.610402 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f579505-ddd8-447b-b05e-69358fcd84db admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:21.640031 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 865/1737] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:33:21 2022] PUT /identity/v3/projects/819cabbf14ab4efab66962d1621ad00c/users/2ef3a193faab46f68e419eece886fd23/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:21.640262 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 872/1737] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:33:21 2022] PUT /identity/v3/projects/4fff94c31fa44e69834b368832ab9a45/users/e5d4739daa8b46618fff36718c462d17/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:21.640357 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f579505-ddd8-447b-b05e-69358fcd84db admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:21.640452 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c894d-29f8-4c36-9984-0799c1d7d6ff None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:21.640452 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c894d-29f8-4c36-9984-0799c1d7d6ff None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:21.640452 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c894d-29f8-4c36-9984-0799c1d7d6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:21.668695 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-629c894d-29f8-4c36-9984-0799c1d7d6ff None None] MFA Rules not processed for user `7cd9c89376814399ab81b7ad849f8484`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:21.670076 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-629c894d-29f8-4c36-9984-0799c1d7d6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:21.712483 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:21.758961 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f579505-ddd8-447b-b05e-69358fcd84db admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:21.830583 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 873/1738] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:21 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:21.834118 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00907d90-48ea-462f-81a6-8c9647aa9718 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:21.834405 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00907d90-48ea-462f-81a6-8c9647aa9718 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:21.834685 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00907d90-48ea-462f-81a6-8c9647aa9718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:21.886563 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-00907d90-48ea-462f-81a6-8c9647aa9718 None None] MFA Rules not processed for user `2ef3a193faab46f68e419eece886fd23`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:21.889672 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-00907d90-48ea-462f-81a6-8c9647aa9718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:21.911852 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f365e78-89cd-447f-8b55-d3236f7b3e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:21.915675 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f365e78-89cd-447f-8b55-d3236f7b3e7c admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:21.917660 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f365e78-89cd-447f-8b55-d3236f7b3e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:21.917791 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f365e78-89cd-447f-8b55-d3236f7b3e7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:21.919299 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f365e78-89cd-447f-8b55-d3236f7b3e7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:21.920969 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f365e78-89cd-447f-8b55-d3236f7b3e7c admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:21.968180 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 866/1739] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:33:21.970602 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8c8ee-1cde-4a1a-bad2-e36b4474dd5b None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:21.970602 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8c8ee-1cde-4a1a-bad2-e36b4474dd5b None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:21.970880 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8c8ee-1cde-4a1a-bad2-e36b4474dd5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:22.048609 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-1fb8c8ee-1cde-4a1a-bad2-e36b4474dd5b None None] MFA Rules not processed for user `e5d4739daa8b46618fff36718c462d17`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:22.051250 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-1fb8c8ee-1cde-4a1a-bad2-e36b4474dd5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:22.088604 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 874/1740] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:21 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:22.099423 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00907d90-48ea-462f-81a6-8c9647aa9718 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:22.102969 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-00907d90-48ea-462f-81a6-8c9647aa9718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:22.201744 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00907d90-48ea-462f-81a6-8c9647aa9718 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:22.214910 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef98bc-f1d5-4adf-9d76-fd9ef8c9228f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:22.217690 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef98bc-f1d5-4adf-9d76-fd9ef8c9228f service cinder] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:22.221802 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef98bc-f1d5-4adf-9d76-fd9ef8c9228f service cinder] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:22.222198 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef98bc-f1d5-4adf-9d76-fd9ef8c9228f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:22.234214 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eef98bc-f1d5-4adf-9d76-fd9ef8c9228f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:22.236034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eef98bc-f1d5-4adf-9d76-fd9ef8c9228f service cinder] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:22.238325 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 867/1741] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:21 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:22.240211 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c23e00-d740-4d39-ada7-394eab44c847 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:22.240211 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c23e00-d740-4d39-ada7-394eab44c847 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:22.240211 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c23e00-d740-4d39-ada7-394eab44c847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:22.254066 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 875/1742] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:33:22 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:33:22.255258 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef98bc-f1d5-4adf-9d76-fd9ef8c9228f service cinder] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:22.280650 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef98bc-f1d5-4adf-9d76-fd9ef8c9228f service cinder] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:22.283872 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-68c23e00-d740-4d39-ada7-394eab44c847 None None] MFA Rules not processed for user `b2e09539caff4d4698cf29c1ea22d365`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:22.287235 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-68c23e00-d740-4d39-ada7-394eab44c847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:22.290708 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875ad556-b103-4d2c-834d-55670d8fa2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:22.291650 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ad556-b103-4d2c-834d-55670d8fa2d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:22.292273 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ad556-b103-4d2c-834d-55670d8fa2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:22.292700 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ad556-b103-4d2c-834d-55670d8fa2d6 admin admin] PATH_INFO: `/v3/projects/454dc6a84056496b9e26c6adf9000cfc/users/9fe991dc33f3440498f895fca6ba8f1c/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:22.295982 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875ad556-b103-4d2c-834d-55670d8fa2d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=454dc6a84056496b9e26c6adf9000cfc, user_id=9fe991dc33f3440498f895fca6ba8f1c, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:22.302414 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875ad556-b103-4d2c-834d-55670d8fa2d6 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:22.314283 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 876/1743] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:33:22 2022] PUT /identity/v3/projects/454dc6a84056496b9e26c6adf9000cfc/users/9fe991dc33f3440498f895fca6ba8f1c/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:22.315317 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875ad556-b103-4d2c-834d-55670d8fa2d6 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:22.317210 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-875ad556-b103-4d2c-834d-55670d8fa2d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:22.389767 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 868/1744] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:22.392797 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c23e00-d740-4d39-ada7-394eab44c847 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:22.407025 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c23e00-d740-4d39-ada7-394eab44c847 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:22.450934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875ad556-b103-4d2c-834d-55670d8fa2d6 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:22.457239 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ba3d6-0cf9-44d6-ab22-72227efd643c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:22.458010 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ba3d6-0cf9-44d6-ab22-72227efd643c service cinder] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:22.458406 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ba3d6-0cf9-44d6-ab22-72227efd643c service cinder] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:22.461454 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ba3d6-0cf9-44d6-ab22-72227efd643c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:22.517473 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7057b92-721b-47f4-a7be-1780313a1d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:22.525084 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7057b92-721b-47f4-a7be-1780313a1d52 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:22.525374 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7057b92-721b-47f4-a7be-1780313a1d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:22.525652 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7057b92-721b-47f4-a7be-1780313a1d52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:22.526516 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7057b92-721b-47f4-a7be-1780313a1d52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:22.528414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7057b92-721b-47f4-a7be-1780313a1d52 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:22.590730 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67ba3d6-0cf9-44d6-ab22-72227efd643c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:22.592555 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67ba3d6-0cf9-44d6-ab22-72227efd643c service cinder] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:22.629909 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 877/1745] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:33:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:33:22.639142 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147bd36-c4f6-4e28-9087-30ec4e9a8a98 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:22.639142 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147bd36-c4f6-4e28-9087-30ec4e9a8a98 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:22.639142 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147bd36-c4f6-4e28-9087-30ec4e9a8a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:22.690747 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-5147bd36-c4f6-4e28-9087-30ec4e9a8a98 None None] MFA Rules not processed for user `9fe991dc33f3440498f895fca6ba8f1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:22.692152 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-5147bd36-c4f6-4e28-9087-30ec4e9a8a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:22.735467 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 869/1746] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:33:22 2022] POST /identity/v3/projects => generated 403 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:33:22.742261 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7057b92-721b-47f4-a7be-1780313a1d52 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:22.825075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:22.852889 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 878/1747] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:22 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:22.853954 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5147bd36-c4f6-4e28-9087-30ec4e9a8a98 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:22.867986 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5147bd36-c4f6-4e28-9087-30ec4e9a8a98 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:22.933508 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7057b92-721b-47f4-a7be-1780313a1d52 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:22.937794 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10166b4-781c-489f-b5e4-099a90c1aaf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:22.938702 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10166b4-781c-489f-b5e4-099a90c1aaf1 service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:22.939932 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10166b4-781c-489f-b5e4-099a90c1aaf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:22.940682 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10166b4-781c-489f-b5e4-099a90c1aaf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:23.015291 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae2b56-1548-4be0-bef7-a5b18954d257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:23.016352 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae2b56-1548-4be0-bef7-a5b18954d257 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:23.016778 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae2b56-1548-4be0-bef7-a5b18954d257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:23.017371 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae2b56-1548-4be0-bef7-a5b18954d257 admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:23.017804 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dae2b56-1548-4be0-bef7-a5b18954d257 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:23.020286 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dae2b56-1548-4be0-bef7-a5b18954d257 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:23.118070 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10166b4-781c-489f-b5e4-099a90c1aaf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:23.119653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10166b4-781c-489f-b5e4-099a90c1aaf1 service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:23.139579 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 879/1748] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:33:22 2022] POST /identity/v3/users => generated 308 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:33:23.140816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae2b56-1548-4be0-bef7-a5b18954d257 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:23.157283 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 870/1749] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:33:22 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:33:23.172790 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae2b56-1548-4be0-bef7-a5b18954d257 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:23.185089 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006776ee-a331-4476-aaa1-4b442fa26671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:23.185967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006776ee-a331-4476-aaa1-4b442fa26671 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:23.186261 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006776ee-a331-4476-aaa1-4b442fa26671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:23.188708 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006776ee-a331-4476-aaa1-4b442fa26671 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:23.191246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006776ee-a331-4476-aaa1-4b442fa26671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:23.192949 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006776ee-a331-4476-aaa1-4b442fa26671 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:23.215331 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 880/1750] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:33:23.219499 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10166b4-781c-489f-b5e4-099a90c1aaf1 service cinder] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:23.242278 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10166b4-781c-489f-b5e4-099a90c1aaf1 service cinder] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:23.255926 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c225d8-1db4-40e4-a7e2-464fd8e7b6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:23.257042 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c225d8-1db4-40e4-a7e2-464fd8e7b6a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:23.258189 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c225d8-1db4-40e4-a7e2-464fd8e7b6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:23.259358 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c225d8-1db4-40e4-a7e2-464fd8e7b6a5 admin admin] PATH_INFO: `/v3/projects/880074b29a694ff9bdd6190da58bb658/users/f108bf8cbccd48938ed19129393f8fb4/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:23.301579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c225d8-1db4-40e4-a7e2-464fd8e7b6a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=880074b29a694ff9bdd6190da58bb658, user_id=f108bf8cbccd48938ed19129393f8fb4, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:23.305754 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c225d8-1db4-40e4-a7e2-464fd8e7b6a5 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:23.438835 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 871/1751] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:33:23 2022] PUT /identity/v3/projects/880074b29a694ff9bdd6190da58bb658/users/f108bf8cbccd48938ed19129393f8fb4/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:23.440471 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006776ee-a331-4476-aaa1-4b442fa26671 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:23.460477 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006776ee-a331-4476-aaa1-4b442fa26671 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:23.575851 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d2043-ccb6-455d-ac61-2a6cd36a21f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:23.576528 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d2043-ccb6-455d-ac61-2a6cd36a21f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:23.576810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d2043-ccb6-455d-ac61-2a6cd36a21f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:23.577085 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d2043-ccb6-455d-ac61-2a6cd36a21f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:23.577857 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d2043-ccb6-455d-ac61-2a6cd36a21f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:23.580223 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d2043-ccb6-455d-ac61-2a6cd36a21f9 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:23.590878 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 881/1752] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:33:23.595207 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c225d8-1db4-40e4-a7e2-464fd8e7b6a5 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:23.606008 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c225d8-1db4-40e4-a7e2-464fd8e7b6a5 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:23.619148 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1de770-8e7b-4017-9f4e-25fab274b70c None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:23.619406 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1de770-8e7b-4017-9f4e-25fab274b70c None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:23.619830 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1de770-8e7b-4017-9f4e-25fab274b70c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:23.622196 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88dd284-f424-47f5-b530-6c1b27f5e681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:23.623029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88dd284-f424-47f5-b530-6c1b27f5e681 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:23.623314 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88dd284-f424-47f5-b530-6c1b27f5e681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:23.623604 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88dd284-f424-47f5-b530-6c1b27f5e681 admin admin] PATH_INFO: `/v3/projects/880074b29a694ff9bdd6190da58bb658/users/f108bf8cbccd48938ed19129393f8fb4/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:23.627488 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88dd284-f424-47f5-b530-6c1b27f5e681 admin admin] RBAC: Authorizing `identity:create_grant(project_id=880074b29a694ff9bdd6190da58bb658, user_id=f108bf8cbccd48938ed19129393f8fb4, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:23.637516 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88dd284-f424-47f5-b530-6c1b27f5e681 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:23.644574 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-fe1de770-8e7b-4017-9f4e-25fab274b70c None None] MFA Rules not processed for user `cd3c04f39d774998b66e6ce920925691`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:23.645585 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-fe1de770-8e7b-4017-9f4e-25fab274b70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:23.664834 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 872/1753] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:33:23 2022] PUT /identity/v3/projects/880074b29a694ff9bdd6190da58bb658/users/f108bf8cbccd48938ed19129393f8fb4/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:23.675161 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030aed21-167d-4d92-93f0-3962291450af None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:23.676733 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030aed21-167d-4d92-93f0-3962291450af None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:23.677074 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030aed21-167d-4d92-93f0-3962291450af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:23.723515 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-030aed21-167d-4d92-93f0-3962291450af None None] MFA Rules not processed for user `f108bf8cbccd48938ed19129393f8fb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:23.725607 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-030aed21-167d-4d92-93f0-3962291450af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:23.752898 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 882/1754] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Jul 7 15:33:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:33:23.788349 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1de770-8e7b-4017-9f4e-25fab274b70c None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:23.850147 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1de770-8e7b-4017-9f4e-25fab274b70c None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:23.853857 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cf09cd-e9ea-4525-af9c-a6941a573596 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:23.854520 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf09cd-e9ea-4525-af9c-a6941a573596 service cinder] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:23.854800 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf09cd-e9ea-4525-af9c-a6941a573596 service cinder] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:23.855342 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf09cd-e9ea-4525-af9c-a6941a573596 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:23.873260 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 873/1755] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:23 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:33:23.883715 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030aed21-167d-4d92-93f0-3962291450af None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:23.902511 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030aed21-167d-4d92-93f0-3962291450af None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:23.907923 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee887b-b826-4475-a389-9d70c4ebe86d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:23.908663 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee887b-b826-4475-a389-9d70c4ebe86d service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:23.909009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee887b-b826-4475-a389-9d70c4ebe86d service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:23.909450 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee887b-b826-4475-a389-9d70c4ebe86d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:23.920934 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:23.924001 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cee887b-b826-4475-a389-9d70c4ebe86d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:23.929139 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cee887b-b826-4475-a389-9d70c4ebe86d service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:23.931662 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cf09cd-e9ea-4525-af9c-a6941a573596 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:23.933337 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cf09cd-e9ea-4525-af9c-a6941a573596 service cinder] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:23.940047 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 874/1756] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:33:23 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:33:23.945647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 883/1757] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:33:23 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:33:24.237661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee887b-b826-4475-a389-9d70c4ebe86d service cinder] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:24.246015 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cf09cd-e9ea-4525-af9c-a6941a573596 service cinder] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:24.248243 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-96cf09cd-e9ea-4525-af9c-a6941a573596 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:24.412517 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee887b-b826-4475-a389-9d70c4ebe86d service cinder] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:24.428601 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5284c0-6cb6-47a6-bc3d-4b33d85e6ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:24.429822 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5284c0-6cb6-47a6-bc3d-4b33d85e6ba6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:24.430113 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5284c0-6cb6-47a6-bc3d-4b33d85e6ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:24.432235 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5284c0-6cb6-47a6-bc3d-4b33d85e6ba6 admin admin] PATH_INFO: `/v3/users/e5d4739daa8b46618fff36718c462d17` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:24.437680 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5284c0-6cb6-47a6-bc3d-4b33d85e6ba6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5d4739daa8b46618fff36718c462d17)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:24.443782 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5284c0-6cb6-47a6-bc3d-4b33d85e6ba6 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:24.488259 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cf09cd-e9ea-4525-af9c-a6941a573596 service cinder] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:24.517061 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d7049-0219-4630-b6d0-9eaaabf81faf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b18033f849b240999d12c1885d2f7b91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:24.517932 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d7049-0219-4630-b6d0-9eaaabf81faf service swift] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:24.521827 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d7049-0219-4630-b6d0-9eaaabf81faf service swift] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:24.522280 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d7049-0219-4630-b6d0-9eaaabf81faf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:24.544187 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999d7049-0219-4630-b6d0-9eaaabf81faf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:24.545844 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999d7049-0219-4630-b6d0-9eaaabf81faf service swift] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:24.574950 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 884/1758] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Jul 7 15:33:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 07 15:33:24.575829 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d7049-0219-4630-b6d0-9eaaabf81faf service swift] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:24.598332 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:24.617433 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d7049-0219-4630-b6d0-9eaaabf81faf service swift] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:24.621549 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5761de-7afc-4451-9a99-ae55d030b367 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:24.624065 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5761de-7afc-4451-9a99-ae55d030b367 service cinder] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:24.624451 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5761de-7afc-4451-9a99-ae55d030b367 service cinder] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:24.624638 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5761de-7afc-4451-9a99-ae55d030b367 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:24.727081 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-bc5284c0-6cb6-47a6-bc3d-4b33d85e6ba6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5d4739daa8b46618fff36718c462d17'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:24.754089 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-bc5284c0-6cb6-47a6-bc3d-4b33d85e6ba6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5d4739daa8b46618fff36718c462d17'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:24.788650 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-bc5284c0-6cb6-47a6-bc3d-4b33d85e6ba6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5d4739daa8b46618fff36718c462d17'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:24.789492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-bc5284c0-6cb6-47a6-bc3d-4b33d85e6ba6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5d4739daa8b46618fff36718c462d17'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:24.836089 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5761de-7afc-4451-9a99-ae55d030b367 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:24.840596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5761de-7afc-4451-9a99-ae55d030b367 service cinder] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:24.861160 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 885/1759] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:33:24 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:33:24.911069 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-bc5284c0-6cb6-47a6-bc3d-4b33d85e6ba6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5d4739daa8b46618fff36718c462d17'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:24.925477 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 875/1760] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:33:24 2022] DELETE /identity/v3/users/e5d4739daa8b46618fff36718c462d17 => generated 0 bytes in 692 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:24.931439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a7363-22c2-4f95-a57d-a82442216c1d None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:24.931775 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a7363-22c2-4f95-a57d-a82442216c1d None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:24.932125 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a7363-22c2-4f95-a57d-a82442216c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:24.985232 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-7c6a7363-22c2-4f95-a57d-a82442216c1d None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:24.990327 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-7c6a7363-22c2-4f95-a57d-a82442216c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:25.094560 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 886/1761] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:25.106081 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5284c0-6cb6-47a6-bc3d-4b33d85e6ba6 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:25.108589 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-bc5284c0-6cb6-47a6-bc3d-4b33d85e6ba6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:25.260498 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5284c0-6cb6-47a6-bc3d-4b33d85e6ba6 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:25.269228 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5502c2e2-f07e-4807-8092-741426845ce8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:25.272911 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502c2e2-f07e-4807-8092-741426845ce8 service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:25.273245 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502c2e2-f07e-4807-8092-741426845ce8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:25.273655 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502c2e2-f07e-4807-8092-741426845ce8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:25.309251 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5502c2e2-f07e-4807-8092-741426845ce8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:25.313414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5502c2e2-f07e-4807-8092-741426845ce8 service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:25.334074 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 876/1762] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:33:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:33:25.742288 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05badd66-7d01-4e66-8cbe-5903386b0210 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:25.743498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05badd66-7d01-4e66-8cbe-5903386b0210 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:25.744069 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05badd66-7d01-4e66-8cbe-5903386b0210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:25.840809 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-05badd66-7d01-4e66-8cbe-5903386b0210 None None] MFA Rules not processed for user `e79efaea155c46ecb9831956e68aec8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:25.847964 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-05badd66-7d01-4e66-8cbe-5903386b0210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:25.862702 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 887/1763] 10.222.0.50 () {58 vars in 1015 bytes} [Thu Jul 7 15:33:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:25.874389 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:25.917783 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5502c2e2-f07e-4807-8092-741426845ce8 service neutron] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:25.943898 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5502c2e2-f07e-4807-8092-741426845ce8 service neutron] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:25.974709 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a0e45-6675-484c-a136-bed8a533093c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:25.975681 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a0e45-6675-484c-a136-bed8a533093c service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:25.976438 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a0e45-6675-484c-a136-bed8a533093c service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:25.976865 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a0e45-6675-484c-a136-bed8a533093c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:25.998702 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a0e45-6675-484c-a136-bed8a533093c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:26.000580 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a0e45-6675-484c-a136-bed8a533093c service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:26.015518 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 877/1764] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:33:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:33:26.333473 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05badd66-7d01-4e66-8cbe-5903386b0210 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:26.336876 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-05badd66-7d01-4e66-8cbe-5903386b0210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:26.365497 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05badd66-7d01-4e66-8cbe-5903386b0210 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:26.375930 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84245e0-87c2-4393-96e1-2c2aacaf0530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:26.377160 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84245e0-87c2-4393-96e1-2c2aacaf0530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:26.377438 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84245e0-87c2-4393-96e1-2c2aacaf0530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:26.377741 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84245e0-87c2-4393-96e1-2c2aacaf0530 admin admin] PATH_INFO: `/v3/projects/4fff94c31fa44e69834b368832ab9a45` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:26.378922 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84245e0-87c2-4393-96e1-2c2aacaf0530 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fff94c31fa44e69834b368832ab9a45)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:26.380904 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84245e0-87c2-4393-96e1-2c2aacaf0530 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:26.566133 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-f84245e0-87c2-4393-96e1-2c2aacaf0530 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fff94c31fa44e69834b368832ab9a45', 'request_id': 'req-f84245e0-87c2-4393-96e1-2c2aacaf0530'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:26.573933 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-f84245e0-87c2-4393-96e1-2c2aacaf0530 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fff94c31fa44e69834b368832ab9a45', 'request_id': 'req-f84245e0-87c2-4393-96e1-2c2aacaf0530'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:26.584689 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-f84245e0-87c2-4393-96e1-2c2aacaf0530 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fff94c31fa44e69834b368832ab9a45', 'request_id': 'req-f84245e0-87c2-4393-96e1-2c2aacaf0530'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:26.705158 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-f84245e0-87c2-4393-96e1-2c2aacaf0530 admin admin] The token cache is being invalidate because project 4fff94c31fa44e69834b368832ab9a45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:33:26.705936 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-f84245e0-87c2-4393-96e1-2c2aacaf0530 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:26.706950 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-f84245e0-87c2-4393-96e1-2c2aacaf0530 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:26.711464 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 888/1765] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:33:26 2022] DELETE /identity/v3/projects/4fff94c31fa44e69834b368832ab9a45 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:26.716216 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a0e45-6675-484c-a136-bed8a533093c service neutron] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:26.737869 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a0e45-6675-484c-a136-bed8a533093c service neutron] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:26.930696 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7287ac3b-cd0b-42bc-b380-e95547f47c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:26.931396 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287ac3b-cd0b-42bc-b380-e95547f47c90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:26.931867 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287ac3b-cd0b-42bc-b380-e95547f47c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:26.932313 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287ac3b-cd0b-42bc-b380-e95547f47c90 admin admin] PATH_INFO: `/v3/users/f108bf8cbccd48938ed19129393f8fb4` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:26.934611 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7287ac3b-cd0b-42bc-b380-e95547f47c90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f108bf8cbccd48938ed19129393f8fb4)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:26.949412 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7287ac3b-cd0b-42bc-b380-e95547f47c90 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:26.969547 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:27.128751 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:27.263472 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-7287ac3b-cd0b-42bc-b380-e95547f47c90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f108bf8cbccd48938ed19129393f8fb4'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:27.290749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-7287ac3b-cd0b-42bc-b380-e95547f47c90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f108bf8cbccd48938ed19129393f8fb4'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:27.337439 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-7287ac3b-cd0b-42bc-b380-e95547f47c90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f108bf8cbccd48938ed19129393f8fb4'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:27.343029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-7287ac3b-cd0b-42bc-b380-e95547f47c90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f108bf8cbccd48938ed19129393f8fb4'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:27.450208 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-7287ac3b-cd0b-42bc-b380-e95547f47c90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f108bf8cbccd48938ed19129393f8fb4'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:27.460480 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 878/1766] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:33:26 2022] DELETE /identity/v3/users/f108bf8cbccd48938ed19129393f8fb4 => generated 0 bytes in 742 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:27.582580 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e66068-1570-459d-9ed0-26603dd8db89 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:27.583046 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e66068-1570-459d-9ed0-26603dd8db89 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:27.583534 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e66068-1570-459d-9ed0-26603dd8db89 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:27.586858 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 889/1767] 10.222.0.25 () {54 vars in 921 bytes} [Thu Jul 7 15:33:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:33:27.602360 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c97710-ca90-4568-a181-32e73ae0d876 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:27.602896 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c97710-ca90-4568-a181-32e73ae0d876 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:27.603288 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c97710-ca90-4568-a181-32e73ae0d876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:27.636596 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-24c97710-ca90-4568-a181-32e73ae0d876 None None] MFA Rules not processed for user `de1632555cb345f2b9213da1925c02f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:27.638155 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-24c97710-ca90-4568-a181-32e73ae0d876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:27.801895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 879/1768] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Jul 7 15:33:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:27.815222 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e66068-1570-459d-9ed0-26603dd8db89 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:27.820117 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-c1e66068-1570-459d-9ed0-26603dd8db89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:27.842216 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e66068-1570-459d-9ed0-26603dd8db89 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:27.858618 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0e2b27-a513-44ef-83b4-44285c171a05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:27.862629 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e2b27-a513-44ef-83b4-44285c171a05 service nova] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:27.862629 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e2b27-a513-44ef-83b4-44285c171a05 service nova] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:27.862629 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e2b27-a513-44ef-83b4-44285c171a05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:27.873137 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0e2b27-a513-44ef-83b4-44285c171a05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:27.874969 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0e2b27-a513-44ef-83b4-44285c171a05 service nova] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:27.888350 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 890/1769] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:33:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:33:28.193293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c97710-ca90-4568-a181-32e73ae0d876 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:28.197695 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-24c97710-ca90-4568-a181-32e73ae0d876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:28.276786 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c97710-ca90-4568-a181-32e73ae0d876 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:28.281412 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddbda1e-a354-4f00-ab8b-ebc8bfe7ccf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:28.282272 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddbda1e-a354-4f00-ab8b-ebc8bfe7ccf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:28.282549 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddbda1e-a354-4f00-ab8b-ebc8bfe7ccf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:28.282822 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddbda1e-a354-4f00-ab8b-ebc8bfe7ccf8 admin admin] PATH_INFO: `/v3/projects/880074b29a694ff9bdd6190da58bb658` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:28.284033 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddbda1e-a354-4f00-ab8b-ebc8bfe7ccf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=880074b29a694ff9bdd6190da58bb658)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:28.285939 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddbda1e-a354-4f00-ab8b-ebc8bfe7ccf8 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:28.288173 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:28.460364 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-2ddbda1e-a354-4f00-ab8b-ebc8bfe7ccf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '880074b29a694ff9bdd6190da58bb658', 'request_id': 'req-2ddbda1e-a354-4f00-ab8b-ebc8bfe7ccf8'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:28.466410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-2ddbda1e-a354-4f00-ab8b-ebc8bfe7ccf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '880074b29a694ff9bdd6190da58bb658', 'request_id': 'req-2ddbda1e-a354-4f00-ab8b-ebc8bfe7ccf8'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:28.474468 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-2ddbda1e-a354-4f00-ab8b-ebc8bfe7ccf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '880074b29a694ff9bdd6190da58bb658', 'request_id': 'req-2ddbda1e-a354-4f00-ab8b-ebc8bfe7ccf8'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:28.506253 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:28.539792 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-2ddbda1e-a354-4f00-ab8b-ebc8bfe7ccf8 admin admin] The token cache is being invalidate because project 880074b29a694ff9bdd6190da58bb658 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:33:28.540087 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-2ddbda1e-a354-4f00-ab8b-ebc8bfe7ccf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:28.541253 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-2ddbda1e-a354-4f00-ab8b-ebc8bfe7ccf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:28.543033 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 880/1770] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:33:28 2022] DELETE /identity/v3/projects/880074b29a694ff9bdd6190da58bb658 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:28.548709 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2f193-f9ca-4e34-96c7-a30237efd9b2 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:28.548986 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2f193-f9ca-4e34-96c7-a30237efd9b2 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:28.549334 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2f193-f9ca-4e34-96c7-a30237efd9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:28.574398 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-5bb2f193-f9ca-4e34-96c7-a30237efd9b2 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:28.575692 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-5bb2f193-f9ca-4e34-96c7-a30237efd9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:28.655464 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 891/1771] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:28.663028 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627f432-57a4-4549-8cd5-827492eda721 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:28.663303 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627f432-57a4-4549-8cd5-827492eda721 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:28.663602 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627f432-57a4-4549-8cd5-827492eda721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:28.685800 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-e627f432-57a4-4549-8cd5-827492eda721 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:28.688806 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-e627f432-57a4-4549-8cd5-827492eda721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:28.699993 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 881/1772] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:28.710645 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb2f193-f9ca-4e34-96c7-a30237efd9b2 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:28.718922 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb2f193-f9ca-4e34-96c7-a30237efd9b2 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:28.725096 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64390d8f-843e-402c-bc83-95698f88cbee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:28.725775 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64390d8f-843e-402c-bc83-95698f88cbee admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:28.726141 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64390d8f-843e-402c-bc83-95698f88cbee admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:28.726423 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64390d8f-843e-402c-bc83-95698f88cbee admin admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:28.727163 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64390d8f-843e-402c-bc83-95698f88cbee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:33:28.727642 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64390d8f-843e-402c-bc83-95698f88cbee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:28.730230 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64390d8f-843e-402c-bc83-95698f88cbee admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:28.745267 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 892/1773] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:33:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:33:28.749244 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e627f432-57a4-4549-8cd5-827492eda721 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:28.761865 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e627f432-57a4-4549-8cd5-827492eda721 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:28.767913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ebc80-3bf6-4bcb-9b23-75a71080afcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:28.774419 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ebc80-3bf6-4bcb-9b23-75a71080afcd admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:28.774419 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ebc80-3bf6-4bcb-9b23-75a71080afcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:28.784588 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ebc80-3bf6-4bcb-9b23-75a71080afcd admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:28.785238 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163ebc80-3bf6-4bcb-9b23-75a71080afcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:28.788872 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163ebc80-3bf6-4bcb-9b23-75a71080afcd admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:28.890328 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64390d8f-843e-402c-bc83-95698f88cbee admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:28.897360 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 882/1774] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:33:28 2022] POST /identity/v3/projects => generated 369 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:33:28.903787 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64390d8f-843e-402c-bc83-95698f88cbee admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:28.988895 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058837a2-39cf-4096-a1b3-fc1f6ae8d24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:28.989681 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058837a2-39cf-4096-a1b3-fc1f6ae8d24d admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:28.989969 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058837a2-39cf-4096-a1b3-fc1f6ae8d24d admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:28.990248 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058837a2-39cf-4096-a1b3-fc1f6ae8d24d admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:28.990939 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058837a2-39cf-4096-a1b3-fc1f6ae8d24d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:28.992885 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058837a2-39cf-4096-a1b3-fc1f6ae8d24d admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:29.054096 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 893/1775] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:33:28 2022] POST /identity/v3/users => generated 285 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:33:29.058226 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ebc80-3bf6-4bcb-9b23-75a71080afcd admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:29.069225 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ebc80-3bf6-4bcb-9b23-75a71080afcd admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:29.074076 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c80bb5-fca2-46cb-ac59-b86b9dc1272b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:29.074790 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c80bb5-fca2-46cb-ac59-b86b9dc1272b admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:29.075073 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c80bb5-fca2-46cb-ac59-b86b9dc1272b admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:29.075353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c80bb5-fca2-46cb-ac59-b86b9dc1272b admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:29.076138 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c80bb5-fca2-46cb-ac59-b86b9dc1272b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:29.077798 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c80bb5-fca2-46cb-ac59-b86b9dc1272b admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:29.091654 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 883/1776] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:33:29.092737 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058837a2-39cf-4096-a1b3-fc1f6ae8d24d admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:29.103483 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058837a2-39cf-4096-a1b3-fc1f6ae8d24d admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:29.110909 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc157da3-f2c2-478a-ab8d-2756d6df1a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:29.110909 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc157da3-f2c2-478a-ab8d-2756d6df1a29 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:29.110909 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc157da3-f2c2-478a-ab8d-2756d6df1a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:29.110909 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc157da3-f2c2-478a-ab8d-2756d6df1a29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:29.111708 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc157da3-f2c2-478a-ab8d-2756d6df1a29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:29.113319 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc157da3-f2c2-478a-ab8d-2756d6df1a29 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:29.124133 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 894/1777] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:33:29.127931 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c80bb5-fca2-46cb-ac59-b86b9dc1272b admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:29.138050 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c80bb5-fca2-46cb-ac59-b86b9dc1272b admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:29.142695 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aeb237-cb49-4cf9-8dd1-552a8a2498e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:29.143608 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aeb237-cb49-4cf9-8dd1-552a8a2498e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:29.143916 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aeb237-cb49-4cf9-8dd1-552a8a2498e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:29.144193 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aeb237-cb49-4cf9-8dd1-552a8a2498e9 admin admin] PATH_INFO: `/v3/projects/c7eeb36d5dfe4c2f94bb20ae446d3f9e/users/6288748c537541e0b51303f55a219a18/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:29.161764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aeb237-cb49-4cf9-8dd1-552a8a2498e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7eeb36d5dfe4c2f94bb20ae446d3f9e, user_id=6288748c537541e0b51303f55a219a18, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:29.164675 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aeb237-cb49-4cf9-8dd1-552a8a2498e9 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:29.271321 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 884/1778] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:33:29 2022] PUT /identity/v3/projects/c7eeb36d5dfe4c2f94bb20ae446d3f9e/users/6288748c537541e0b51303f55a219a18/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:33:29.283345 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419c8b8-2490-4117-829d-05a303b059a0 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:29.283802 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419c8b8-2490-4117-829d-05a303b059a0 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:29.283954 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419c8b8-2490-4117-829d-05a303b059a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:29.322224 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-4419c8b8-2490-4117-829d-05a303b059a0 None None] MFA Rules not processed for user `6288748c537541e0b51303f55a219a18`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:29.323042 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-4419c8b8-2490-4117-829d-05a303b059a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:29.397974 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 895/1779] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:29 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:29.402763 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aeb237-cb49-4cf9-8dd1-552a8a2498e9 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:29.469242 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aeb237-cb49-4cf9-8dd1-552a8a2498e9 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:29.476146 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f36a4-50b5-43f0-a3f2-3a62497cf4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:29.476961 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f36a4-50b5-43f0-a3f2-3a62497cf4f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:29.477378 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f36a4-50b5-43f0-a3f2-3a62497cf4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:29.477717 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f36a4-50b5-43f0-a3f2-3a62497cf4f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:29.478537 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8f36a4-50b5-43f0-a3f2-3a62497cf4f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:29.482221 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8f36a4-50b5-43f0-a3f2-3a62497cf4f5 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:29.510215 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:29.645734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 885/1780] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:33:29 2022] POST /identity/v3/projects => generated 369 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:33:29.649899 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4419c8b8-2490-4117-829d-05a303b059a0 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:29.662909 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4419c8b8-2490-4117-829d-05a303b059a0 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:29.750279 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de76d66-5480-4b56-8801-8e6927d0e338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:29.751084 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de76d66-5480-4b56-8801-8e6927d0e338 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:29.751416 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de76d66-5480-4b56-8801-8e6927d0e338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:29.751765 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de76d66-5480-4b56-8801-8e6927d0e338 admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:29.752473 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de76d66-5480-4b56-8801-8e6927d0e338 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:29.754844 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de76d66-5480-4b56-8801-8e6927d0e338 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:29.831094 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:29.867036 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f36a4-50b5-43f0-a3f2-3a62497cf4f5 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:29.869721 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 896/1781] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:33:29 2022] POST /identity/v3/users => generated 291 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:33:29.870888 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-dc8f36a4-50b5-43f0-a3f2-3a62497cf4f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:29.879914 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de76d66-5480-4b56-8801-8e6927d0e338 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:29.911853 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de76d66-5480-4b56-8801-8e6927d0e338 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:29.917363 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b6cc44-c7d1-4b25-9db0-dbe2b17cb591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:29.918342 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6cc44-c7d1-4b25-9db0-dbe2b17cb591 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:29.918771 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6cc44-c7d1-4b25-9db0-dbe2b17cb591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:29.919186 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6cc44-c7d1-4b25-9db0-dbe2b17cb591 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:29.920159 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b6cc44-c7d1-4b25-9db0-dbe2b17cb591 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:29.921969 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b6cc44-c7d1-4b25-9db0-dbe2b17cb591 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:29.936480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 897/1782] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:33:29.950004 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b6cc44-c7d1-4b25-9db0-dbe2b17cb591 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:29.982897 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b6cc44-c7d1-4b25-9db0-dbe2b17cb591 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:29.989698 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f36a4-50b5-43f0-a3f2-3a62497cf4f5 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:30.002524 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0beb7-ba12-4372-a372-dcf8fdff91c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:30.003822 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd619bc-38f9-4bac-89e2-42df6351f32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:30.004524 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0beb7-ba12-4372-a372-dcf8fdff91c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:30.004812 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0beb7-ba12-4372-a372-dcf8fdff91c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:30.004917 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd619bc-38f9-4bac-89e2-42df6351f32d admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:30.005092 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0beb7-ba12-4372-a372-dcf8fdff91c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:30.005199 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd619bc-38f9-4bac-89e2-42df6351f32d admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:30.005429 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd619bc-38f9-4bac-89e2-42df6351f32d admin admin] PATH_INFO: `/v3/projects/8f71cb6e8e08479099d23e425063e4c7/users/f1377baba4394035b0152d71d0c75eb8/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:30.009406 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-7ee0beb7-ba12-4372-a372-dcf8fdff91c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:30.029762 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd619bc-38f9-4bac-89e2-42df6351f32d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f71cb6e8e08479099d23e425063e4c7, user_id=f1377baba4394035b0152d71d0c75eb8, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:30.032816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd619bc-38f9-4bac-89e2-42df6351f32d admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:30.123627 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 898/1783] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:33:29 2022] PUT /identity/v3/projects/8f71cb6e8e08479099d23e425063e4c7/users/f1377baba4394035b0152d71d0c75eb8/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:30.128003 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd619bc-38f9-4bac-89e2-42df6351f32d admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:30.137522 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee0beb7-ba12-4372-a372-dcf8fdff91c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:30.139281 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee0beb7-ba12-4372-a372-dcf8fdff91c4 service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:30.150620 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 886/1784] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:33:29 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 15:33:30.309731 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd619bc-38f9-4bac-89e2-42df6351f32d admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:30.321283 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609143da-304a-4321-845b-08ee1ed593df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:30.322416 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609143da-304a-4321-845b-08ee1ed593df admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:30.323340 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609143da-304a-4321-845b-08ee1ed593df admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:30.324226 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609143da-304a-4321-845b-08ee1ed593df admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:30.325113 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609143da-304a-4321-845b-08ee1ed593df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:30.328122 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609143da-304a-4321-845b-08ee1ed593df admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:30.352391 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0beb7-ba12-4372-a372-dcf8fdff91c4 service cinder] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:30.353158 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 899/1785] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:33:30.374207 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0beb7-ba12-4372-a372-dcf8fdff91c4 service cinder] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:30.394417 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a769cc2-e2fc-4469-a9c7-858961ce0f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:30.395878 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a769cc2-e2fc-4469-a9c7-858961ce0f66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:30.395878 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a769cc2-e2fc-4469-a9c7-858961ce0f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:30.395878 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a769cc2-e2fc-4469-a9c7-858961ce0f66 admin admin] PATH_INFO: `/v3/projects/8f71cb6e8e08479099d23e425063e4c7/users/f1377baba4394035b0152d71d0c75eb8/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:30.404685 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a769cc2-e2fc-4469-a9c7-858961ce0f66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f71cb6e8e08479099d23e425063e4c7, user_id=f1377baba4394035b0152d71d0c75eb8, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:30.416662 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a769cc2-e2fc-4469-a9c7-858961ce0f66 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:30.466461 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 887/1786] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:33:30 2022] PUT /identity/v3/projects/8f71cb6e8e08479099d23e425063e4c7/users/f1377baba4394035b0152d71d0c75eb8/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:33:30.468149 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208da72-298d-435f-af86-e6f951a3c30c None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:30.469150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208da72-298d-435f-af86-e6f951a3c30c None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:30.469732 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208da72-298d-435f-af86-e6f951a3c30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:30.574267 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:30.598814 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-1208da72-298d-435f-af86-e6f951a3c30c None None] MFA Rules not processed for user `f1377baba4394035b0152d71d0c75eb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:30.600090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-1208da72-298d-435f-af86-e6f951a3c30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:30.786439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 900/1787] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:30 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:31.041054 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a769cc2-e2fc-4469-a9c7-858961ce0f66 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:31.100860 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:31.114306 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a769cc2-e2fc-4469-a9c7-858961ce0f66 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:31.118614 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff2479-1205-46a3-9c98-121271a63e47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:31.120236 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff2479-1205-46a3-9c98-121271a63e47 service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:31.121254 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff2479-1205-46a3-9c98-121271a63e47 service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:31.121624 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff2479-1205-46a3-9c98-121271a63e47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:31.134864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ff2479-1205-46a3-9c98-121271a63e47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:31.136402 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ff2479-1205-46a3-9c98-121271a63e47 service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:31.150757 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 888/1788] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:33:31 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:33:31.520269 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1208da72-298d-435f-af86-e6f951a3c30c None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:31.533843 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1208da72-298d-435f-af86-e6f951a3c30c None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:31.541413 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49015a1a-4da4-4f11-a0c4-38fa465b596f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:31.542217 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49015a1a-4da4-4f11-a0c4-38fa465b596f service cinder] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:31.542578 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49015a1a-4da4-4f11-a0c4-38fa465b596f service cinder] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:31.542952 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49015a1a-4da4-4f11-a0c4-38fa465b596f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:31.619787 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49015a1a-4da4-4f11-a0c4-38fa465b596f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:31.621427 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49015a1a-4da4-4f11-a0c4-38fa465b596f service cinder] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:31.634055 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 901/1789] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:33:31 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:33:31.720040 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:32.108115 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:33.534273 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:33.913772 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:34.039512 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:34.892423 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:35.056467 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:35.426218 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:36.485243 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:36.710240 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:37.718433 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff2479-1205-46a3-9c98-121271a63e47 service cinder] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:37.782154 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff2479-1205-46a3-9c98-121271a63e47 service cinder] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:37.786865 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08836ac1-c215-40fb-9957-838275b1bd05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:37.787769 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08836ac1-c215-40fb-9957-838275b1bd05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:37.788049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08836ac1-c215-40fb-9957-838275b1bd05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:37.788347 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08836ac1-c215-40fb-9957-838275b1bd05 admin admin] PATH_INFO: `/v3/users/6288748c537541e0b51303f55a219a18` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:37.789709 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08836ac1-c215-40fb-9957-838275b1bd05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6288748c537541e0b51303f55a219a18)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:37.791638 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08836ac1-c215-40fb-9957-838275b1bd05 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:37.819171 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:37.933754 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:37.936358 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-08836ac1-c215-40fb-9957-838275b1bd05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6288748c537541e0b51303f55a219a18'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:37.943648 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-08836ac1-c215-40fb-9957-838275b1bd05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6288748c537541e0b51303f55a219a18'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:37.955650 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-08836ac1-c215-40fb-9957-838275b1bd05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6288748c537541e0b51303f55a219a18'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:37.963612 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-08836ac1-c215-40fb-9957-838275b1bd05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6288748c537541e0b51303f55a219a18'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:38.039093 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-08836ac1-c215-40fb-9957-838275b1bd05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6288748c537541e0b51303f55a219a18'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:38.047161 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 889/1790] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:33:37 2022] DELETE /identity/v3/users/6288748c537541e0b51303f55a219a18 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:38.051828 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862e7926-d927-4c33-98f4-256e77a5a091 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:38.055374 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862e7926-d927-4c33-98f4-256e77a5a091 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:38.055374 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862e7926-d927-4c33-98f4-256e77a5a091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:38.072994 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-862e7926-d927-4c33-98f4-256e77a5a091 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:38.074324 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-862e7926-d927-4c33-98f4-256e77a5a091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:38.134116 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 902/1791] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:38.143693 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08836ac1-c215-40fb-9957-838275b1bd05 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:38.145105 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-08836ac1-c215-40fb-9957-838275b1bd05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:38.201331 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08836ac1-c215-40fb-9957-838275b1bd05 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:38.205407 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4272ab-a820-4813-bb2a-afc44d241c7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:38.206226 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4272ab-a820-4813-bb2a-afc44d241c7b service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:38.206507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4272ab-a820-4813-bb2a-afc44d241c7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:38.206780 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4272ab-a820-4813-bb2a-afc44d241c7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:38.215460 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4272ab-a820-4813-bb2a-afc44d241c7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:38.216893 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4272ab-a820-4813-bb2a-afc44d241c7b service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:38.227014 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 890/1792] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:33:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:33:39.305906 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862e7926-d927-4c33-98f4-256e77a5a091 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:39.305906 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-862e7926-d927-4c33-98f4-256e77a5a091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:39.328118 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862e7926-d927-4c33-98f4-256e77a5a091 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:39.333416 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36218805-ea82-447c-928b-b97544510695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:39.334291 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36218805-ea82-447c-928b-b97544510695 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:39.334612 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36218805-ea82-447c-928b-b97544510695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:39.335074 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36218805-ea82-447c-928b-b97544510695 admin admin] PATH_INFO: `/v3/projects/c7eeb36d5dfe4c2f94bb20ae446d3f9e` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:39.336612 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36218805-ea82-447c-928b-b97544510695 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7eeb36d5dfe4c2f94bb20ae446d3f9e)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:39.338885 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36218805-ea82-447c-928b-b97544510695 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:39.384264 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:39.501147 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-36218805-ea82-447c-928b-b97544510695 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7eeb36d5dfe4c2f94bb20ae446d3f9e', 'request_id': 'req-36218805-ea82-447c-928b-b97544510695'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:39.507747 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-36218805-ea82-447c-928b-b97544510695 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7eeb36d5dfe4c2f94bb20ae446d3f9e', 'request_id': 'req-36218805-ea82-447c-928b-b97544510695'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:39.514316 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-36218805-ea82-447c-928b-b97544510695 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7eeb36d5dfe4c2f94bb20ae446d3f9e', 'request_id': 'req-36218805-ea82-447c-928b-b97544510695'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:39.657737 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-36218805-ea82-447c-928b-b97544510695 admin admin] The token cache is being invalidate because project c7eeb36d5dfe4c2f94bb20ae446d3f9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:33:39.658354 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-36218805-ea82-447c-928b-b97544510695 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:39.659761 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-36218805-ea82-447c-928b-b97544510695 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:39.665953 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 903/1793] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:33:39 2022] DELETE /identity/v3/projects/c7eeb36d5dfe4c2f94bb20ae446d3f9e => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:39.668257 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4272ab-a820-4813-bb2a-afc44d241c7b service neutron] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:39.681930 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4272ab-a820-4813-bb2a-afc44d241c7b service neutron] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:39.779138 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cfe76f-323e-404c-88eb-096194e733a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:39.786122 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfe76f-323e-404c-88eb-096194e733a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:39.786122 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfe76f-323e-404c-88eb-096194e733a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:39.786122 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfe76f-323e-404c-88eb-096194e733a5 admin admin] PATH_INFO: `/v3/users/f1377baba4394035b0152d71d0c75eb8` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:39.786122 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cfe76f-323e-404c-88eb-096194e733a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1377baba4394035b0152d71d0c75eb8)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:39.786122 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cfe76f-323e-404c-88eb-096194e733a5 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:40.072041 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-d5cfe76f-323e-404c-88eb-096194e733a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1377baba4394035b0152d71d0c75eb8'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:40.082566 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-d5cfe76f-323e-404c-88eb-096194e733a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1377baba4394035b0152d71d0c75eb8'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:40.093401 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:40.099473 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-d5cfe76f-323e-404c-88eb-096194e733a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1377baba4394035b0152d71d0c75eb8'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:40.100257 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-d5cfe76f-323e-404c-88eb-096194e733a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1377baba4394035b0152d71d0c75eb8'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:40.191745 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-d5cfe76f-323e-404c-88eb-096194e733a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1377baba4394035b0152d71d0c75eb8'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:40.207612 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 891/1794] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:33:39 2022] DELETE /identity/v3/users/f1377baba4394035b0152d71d0c75eb8 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:40.849208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36218805-ea82-447c-928b-b97544510695 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:40.853171 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-36218805-ea82-447c-928b-b97544510695 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:40.867815 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36218805-ea82-447c-928b-b97544510695 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:40.939042 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242a6fc4-b844-4ee1-ae9a-e54498d8a9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:40.942959 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a6fc4-b844-4ee1-ae9a-e54498d8a9b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:40.944291 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a6fc4-b844-4ee1-ae9a-e54498d8a9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:40.944624 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a6fc4-b844-4ee1-ae9a-e54498d8a9b3 admin admin] PATH_INFO: `/v3/projects/8f71cb6e8e08479099d23e425063e4c7` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:40.946073 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242a6fc4-b844-4ee1-ae9a-e54498d8a9b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f71cb6e8e08479099d23e425063e4c7)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:40.948121 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242a6fc4-b844-4ee1-ae9a-e54498d8a9b3 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:41.078782 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-242a6fc4-b844-4ee1-ae9a-e54498d8a9b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f71cb6e8e08479099d23e425063e4c7', 'request_id': 'req-242a6fc4-b844-4ee1-ae9a-e54498d8a9b3'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:41.084328 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-242a6fc4-b844-4ee1-ae9a-e54498d8a9b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f71cb6e8e08479099d23e425063e4c7', 'request_id': 'req-242a6fc4-b844-4ee1-ae9a-e54498d8a9b3'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:41.090718 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-242a6fc4-b844-4ee1-ae9a-e54498d8a9b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f71cb6e8e08479099d23e425063e4c7', 'request_id': 'req-242a6fc4-b844-4ee1-ae9a-e54498d8a9b3'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:41.198508 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-242a6fc4-b844-4ee1-ae9a-e54498d8a9b3 admin admin] The token cache is being invalidate because project 8f71cb6e8e08479099d23e425063e4c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:33:41.198805 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-242a6fc4-b844-4ee1-ae9a-e54498d8a9b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:41.202618 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-242a6fc4-b844-4ee1-ae9a-e54498d8a9b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:41.202618 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 904/1795] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:33:40 2022] DELETE /identity/v3/projects/8f71cb6e8e08479099d23e425063e4c7 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:33:41.206906 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569641a-e4c7-440c-b0ea-8632d6d64dbd None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:41.207402 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569641a-e4c7-440c-b0ea-8632d6d64dbd None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:41.207609 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569641a-e4c7-440c-b0ea-8632d6d64dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:41.243027 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-4569641a-e4c7-440c-b0ea-8632d6d64dbd None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:41.244827 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-4569641a-e4c7-440c-b0ea-8632d6d64dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:41.329557 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 892/1796] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:33:41.343046 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cbc2e9-160b-4e8b-a85c-675a3a34caa3 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:41.343637 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cbc2e9-160b-4e8b-a85c-675a3a34caa3 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:41.344489 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cbc2e9-160b-4e8b-a85c-675a3a34caa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:41.392891 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-79cbc2e9-160b-4e8b-a85c-675a3a34caa3 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:41.394892 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-79cbc2e9-160b-4e8b-a85c-675a3a34caa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:41.406446 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 905/1797] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:41.410659 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4569641a-e4c7-440c-b0ea-8632d6d64dbd None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:41.422121 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4569641a-e4c7-440c-b0ea-8632d6d64dbd None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:41.428671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da60509-a076-44f2-8b1b-5de7848c91bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:41.431503 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da60509-a076-44f2-8b1b-5de7848c91bc admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:41.431503 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da60509-a076-44f2-8b1b-5de7848c91bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:41.431503 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da60509-a076-44f2-8b1b-5de7848c91bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:41.431503 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da60509-a076-44f2-8b1b-5de7848c91bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:33:41.431503 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da60509-a076-44f2-8b1b-5de7848c91bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:41.433134 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da60509-a076-44f2-8b1b-5de7848c91bc admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:41.453447 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cbc2e9-160b-4e8b-a85c-675a3a34caa3 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:41.460389 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 893/1798] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:33:41.464634 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cbc2e9-160b-4e8b-a85c-675a3a34caa3 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:41.470609 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe84394-e91a-406e-9aa1-887a34b1795a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:41.471396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe84394-e91a-406e-9aa1-887a34b1795a admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:41.471828 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe84394-e91a-406e-9aa1-887a34b1795a admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:41.472232 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe84394-e91a-406e-9aa1-887a34b1795a admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:41.473202 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe84394-e91a-406e-9aa1-887a34b1795a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:41.475425 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe84394-e91a-406e-9aa1-887a34b1795a admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:41.605762 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 906/1799] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:33:41 2022] POST /identity/v3/projects => generated 383 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:33:41.606993 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da60509-a076-44f2-8b1b-5de7848c91bc admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:41.620678 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da60509-a076-44f2-8b1b-5de7848c91bc admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:41.709688 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4687e6e2-d1ed-409e-bb7e-6c1d252c8b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:41.711203 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4687e6e2-d1ed-409e-bb7e-6c1d252c8b61 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:41.712177 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4687e6e2-d1ed-409e-bb7e-6c1d252c8b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:41.712177 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4687e6e2-d1ed-409e-bb7e-6c1d252c8b61 admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:41.713045 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4687e6e2-d1ed-409e-bb7e-6c1d252c8b61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:41.715447 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4687e6e2-d1ed-409e-bb7e-6c1d252c8b61 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:41.832939 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 894/1800] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:33:41 2022] POST /identity/v3/users => generated 292 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:33:41.837384 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe84394-e91a-406e-9aa1-887a34b1795a admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:41.851559 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe84394-e91a-406e-9aa1-887a34b1795a admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:41.859938 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f3dc74-0564-4002-9432-a682bcfc317d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:41.860936 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3dc74-0564-4002-9432-a682bcfc317d admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:41.861376 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3dc74-0564-4002-9432-a682bcfc317d admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:41.862507 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3dc74-0564-4002-9432-a682bcfc317d admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:41.863806 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f3dc74-0564-4002-9432-a682bcfc317d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:41.865782 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f3dc74-0564-4002-9432-a682bcfc317d admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:41.885749 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 907/1801] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:33:41.886209 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4687e6e2-d1ed-409e-bb7e-6c1d252c8b61 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:41.910775 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4687e6e2-d1ed-409e-bb7e-6c1d252c8b61 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:41.918878 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9915bf56-304a-4fbe-bdfa-e91b27850622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:41.919723 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915bf56-304a-4fbe-bdfa-e91b27850622 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:41.919993 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915bf56-304a-4fbe-bdfa-e91b27850622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:41.920268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915bf56-304a-4fbe-bdfa-e91b27850622 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:41.921049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9915bf56-304a-4fbe-bdfa-e91b27850622 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:41.924115 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9915bf56-304a-4fbe-bdfa-e91b27850622 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:41.941898 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f3dc74-0564-4002-9432-a682bcfc317d admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:41.949896 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:41.950061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 895/1802] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:33:41.974949 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f3dc74-0564-4002-9432-a682bcfc317d admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:41.982472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1cdb1e-ab61-486b-a67b-74508c2ac1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:41.984011 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1cdb1e-ab61-486b-a67b-74508c2ac1da admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:41.984488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1cdb1e-ab61-486b-a67b-74508c2ac1da admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:41.984835 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1cdb1e-ab61-486b-a67b-74508c2ac1da admin admin] PATH_INFO: `/v3/projects/ce9aa7436e5d407e862975e5c8b0ef81/users/bf10fbe85ca64394b56bd5b81b012803/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:42.003486 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1cdb1e-ab61-486b-a67b-74508c2ac1da admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce9aa7436e5d407e862975e5c8b0ef81, user_id=bf10fbe85ca64394b56bd5b81b012803, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:42.006813 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1cdb1e-ab61-486b-a67b-74508c2ac1da admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:42.121554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 908/1803] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:33:41 2022] PUT /identity/v3/projects/ce9aa7436e5d407e862975e5c8b0ef81/users/bf10fbe85ca64394b56bd5b81b012803/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:42.138936 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e2716-7710-4481-ad9d-fc8bbd2efae5 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:42.139484 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e2716-7710-4481-ad9d-fc8bbd2efae5 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:42.139811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e2716-7710-4481-ad9d-fc8bbd2efae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:42.189632 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-3b9e2716-7710-4481-ad9d-fc8bbd2efae5 None None] MFA Rules not processed for user `bf10fbe85ca64394b56bd5b81b012803`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:42.202028 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-3b9e2716-7710-4481-ad9d-fc8bbd2efae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:42.265099 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:42.297592 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 896/1804] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:42.307691 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1cdb1e-ab61-486b-a67b-74508c2ac1da admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:42.447071 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1cdb1e-ab61-486b-a67b-74508c2ac1da admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:42.456829 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab86e2-6e41-4248-b274-5711725ec782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:42.456829 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab86e2-6e41-4248-b274-5711725ec782 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:42.456829 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab86e2-6e41-4248-b274-5711725ec782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:42.456829 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab86e2-6e41-4248-b274-5711725ec782 admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:42.457337 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fab86e2-6e41-4248-b274-5711725ec782 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:42.457722 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fab86e2-6e41-4248-b274-5711725ec782 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:42.592175 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 909/1805] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:33:42 2022] POST /identity/v3/projects => generated 387 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:33:42.596944 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e2716-7710-4481-ad9d-fc8bbd2efae5 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:42.618734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e2716-7710-4481-ad9d-fc8bbd2efae5 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:42.721370 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151017f-413d-40fa-8868-44f0eea53d16 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:42.721669 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151017f-413d-40fa-8868-44f0eea53d16 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:42.721953 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151017f-413d-40fa-8868-44f0eea53d16 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:42.723341 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 910/1806] 10.222.0.25 () {54 vars in 921 bytes} [Thu Jul 7 15:33:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:33:42.737067 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ee7de-5bff-44f2-b69e-19612265acc2 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:42.737360 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ee7de-5bff-44f2-b69e-19612265acc2 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:42.737644 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ee7de-5bff-44f2-b69e-19612265acc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:42.751345 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f07557-af71-4d48-82e9-dc787f799290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:42.752270 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f07557-af71-4d48-82e9-dc787f799290 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:42.752763 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f07557-af71-4d48-82e9-dc787f799290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:42.753135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f07557-af71-4d48-82e9-dc787f799290 admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:42.754583 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f07557-af71-4d48-82e9-dc787f799290 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:42.756995 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f07557-af71-4d48-82e9-dc787f799290 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:42.795870 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-1a5ee7de-5bff-44f2-b69e-19612265acc2 None None] MFA Rules not processed for user `de1632555cb345f2b9213da1925c02f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:42.797361 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-1a5ee7de-5bff-44f2-b69e-19612265acc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:42.908771 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 911/1807] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Jul 7 15:33:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:42.927002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 897/1808] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:33:42 2022] POST /identity/v3/users => generated 300 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:33:42.935471 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ee7de-5bff-44f2-b69e-19612265acc2 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:42.942271 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f07557-af71-4d48-82e9-dc787f799290 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:42.944250 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-55f07557-af71-4d48-82e9-dc787f799290 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:42.970852 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ee7de-5bff-44f2-b69e-19612265acc2 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:42.975709 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f07557-af71-4d48-82e9-dc787f799290 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:42.982352 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb2173-cd80-4827-9635-16bed2a594c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:42.983091 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb2173-cd80-4827-9635-16bed2a594c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:42.983546 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb2173-cd80-4827-9635-16bed2a594c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:42.983858 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb2173-cd80-4827-9635-16bed2a594c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:42.984607 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eb2173-cd80-4827-9635-16bed2a594c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:42.986222 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eb2173-cd80-4827-9635-16bed2a594c5 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:42.986894 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d06114-a5cb-40f8-98a3-026b702f4cec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:42.987753 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d06114-a5cb-40f8-98a3-026b702f4cec service nova] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:42.994248 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d06114-a5cb-40f8-98a3-026b702f4cec service nova] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:42.994248 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d06114-a5cb-40f8-98a3-026b702f4cec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:42.997776 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 912/1809] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:33:43.000596 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d06114-a5cb-40f8-98a3-026b702f4cec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:43.001826 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb2173-cd80-4827-9635-16bed2a594c5 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:43.002674 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d06114-a5cb-40f8-98a3-026b702f4cec service nova] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:43.012270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb2173-cd80-4827-9635-16bed2a594c5 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:43.013965 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 898/1810] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:33:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 15:33:43.025332 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecca7ea-90a5-49b2-9957-3dd7a678e99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:43.028594 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecca7ea-90a5-49b2-9957-3dd7a678e99c admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:43.028594 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecca7ea-90a5-49b2-9957-3dd7a678e99c admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:43.028594 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecca7ea-90a5-49b2-9957-3dd7a678e99c admin admin] PATH_INFO: `/v3/projects/5d04d85c8a844d5382d6073e0dae5a58/users/6551d84eefb84108a4fd0e208ead0a6f/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:43.046460 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecca7ea-90a5-49b2-9957-3dd7a678e99c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d04d85c8a844d5382d6073e0dae5a58, user_id=6551d84eefb84108a4fd0e208ead0a6f, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:43.049671 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecca7ea-90a5-49b2-9957-3dd7a678e99c admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:43.166145 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 913/1811] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:33:43 2022] PUT /identity/v3/projects/5d04d85c8a844d5382d6073e0dae5a58/users/6551d84eefb84108a4fd0e208ead0a6f/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:33:43.169950 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d06114-a5cb-40f8-98a3-026b702f4cec service nova] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:43.212263 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d06114-a5cb-40f8-98a3-026b702f4cec service nova] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:43.289043 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3206c2f7-f8e1-4903-bb51-d61afbfda740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:43.289724 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3206c2f7-f8e1-4903-bb51-d61afbfda740 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:43.290012 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3206c2f7-f8e1-4903-bb51-d61afbfda740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:43.290291 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3206c2f7-f8e1-4903-bb51-d61afbfda740 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:43.291038 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3206c2f7-f8e1-4903-bb51-d61afbfda740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:43.294724 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3206c2f7-f8e1-4903-bb51-d61afbfda740 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:43.314627 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 899/1812] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:43 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:33:43.315816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecca7ea-90a5-49b2-9957-3dd7a678e99c admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:43.337488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecca7ea-90a5-49b2-9957-3dd7a678e99c admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:43.348367 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595673f8-1bee-448d-8c12-dbb7073e6abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:43.349288 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595673f8-1bee-448d-8c12-dbb7073e6abe admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:43.349862 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595673f8-1bee-448d-8c12-dbb7073e6abe admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:43.349862 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595673f8-1bee-448d-8c12-dbb7073e6abe admin admin] PATH_INFO: `/v3/projects/5d04d85c8a844d5382d6073e0dae5a58/users/6551d84eefb84108a4fd0e208ead0a6f/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:43.356456 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595673f8-1bee-448d-8c12-dbb7073e6abe admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d04d85c8a844d5382d6073e0dae5a58, user_id=6551d84eefb84108a4fd0e208ead0a6f, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:43.361691 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595673f8-1bee-448d-8c12-dbb7073e6abe admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:43.375137 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 914/1813] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:33:43 2022] PUT /identity/v3/projects/5d04d85c8a844d5382d6073e0dae5a58/users/6551d84eefb84108a4fd0e208ead0a6f/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:43.388284 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e50f8-d6e0-44b8-b11b-27ba45dab684 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:43.388652 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e50f8-d6e0-44b8-b11b-27ba45dab684 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:43.389006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e50f8-d6e0-44b8-b11b-27ba45dab684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:43.430463 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-067e50f8-d6e0-44b8-b11b-27ba45dab684 None None] MFA Rules not processed for user `6551d84eefb84108a4fd0e208ead0a6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:43.432407 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-067e50f8-d6e0-44b8-b11b-27ba45dab684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:43.514336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 900/1814] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:43 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:43.518842 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:43.525775 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595673f8-1bee-448d-8c12-dbb7073e6abe admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:43.527435 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-595673f8-1bee-448d-8c12-dbb7073e6abe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:43.537148 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:43.587961 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595673f8-1bee-448d-8c12-dbb7073e6abe admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:43.591670 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058654bf-5220-4441-8c71-abe4b4632acf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:43.592311 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058654bf-5220-4441-8c71-abe4b4632acf service cinder] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:43.592617 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058654bf-5220-4441-8c71-abe4b4632acf service cinder] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:43.592915 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058654bf-5220-4441-8c71-abe4b4632acf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:43.602169 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058654bf-5220-4441-8c71-abe4b4632acf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:43.608686 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058654bf-5220-4441-8c71-abe4b4632acf service cinder] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:43.613816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 915/1815] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:33:43 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:33:43.686005 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e50f8-d6e0-44b8-b11b-27ba45dab684 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:43.750335 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e50f8-d6e0-44b8-b11b-27ba45dab684 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:43.754916 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734480e4-955d-405e-aba9-02732e4be08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:43.755825 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734480e4-955d-405e-aba9-02732e4be08a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:43.756100 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734480e4-955d-405e-aba9-02732e4be08a admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:43.756370 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734480e4-955d-405e-aba9-02732e4be08a admin admin] PATH_INFO: `/v3/users/bf10fbe85ca64394b56bd5b81b012803` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:43.757595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734480e4-955d-405e-aba9-02732e4be08a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf10fbe85ca64394b56bd5b81b012803)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:43.759538 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734480e4-955d-405e-aba9-02732e4be08a admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:43.946328 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-734480e4-955d-405e-aba9-02732e4be08a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf10fbe85ca64394b56bd5b81b012803'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:43.953642 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-734480e4-955d-405e-aba9-02732e4be08a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf10fbe85ca64394b56bd5b81b012803'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:43.966279 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-734480e4-955d-405e-aba9-02732e4be08a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf10fbe85ca64394b56bd5b81b012803'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:43.967023 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-734480e4-955d-405e-aba9-02732e4be08a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf10fbe85ca64394b56bd5b81b012803'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:44.047174 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-734480e4-955d-405e-aba9-02732e4be08a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf10fbe85ca64394b56bd5b81b012803'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:44.054804 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 901/1816] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:33:43 2022] DELETE /identity/v3/users/bf10fbe85ca64394b56bd5b81b012803 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:44.060923 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a73b861-530c-46f0-bf07-4a5022f442b1 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:44.061928 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a73b861-530c-46f0-bf07-4a5022f442b1 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:44.062930 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a73b861-530c-46f0-bf07-4a5022f442b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:44.090082 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-9a73b861-530c-46f0-bf07-4a5022f442b1 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:44.091383 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-9a73b861-530c-46f0-bf07-4a5022f442b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:44.162925 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 916/1817] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:44.172662 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734480e4-955d-405e-aba9-02732e4be08a admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:44.176901 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-734480e4-955d-405e-aba9-02732e4be08a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:44.239822 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734480e4-955d-405e-aba9-02732e4be08a admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:44.244455 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9509eb3-6a1c-458b-a3ff-c79923b78c95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:44.245113 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9509eb3-6a1c-458b-a3ff-c79923b78c95 service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:44.245397 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9509eb3-6a1c-458b-a3ff-c79923b78c95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:44.245680 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9509eb3-6a1c-458b-a3ff-c79923b78c95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:44.255766 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9509eb3-6a1c-458b-a3ff-c79923b78c95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:44.257180 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9509eb3-6a1c-458b-a3ff-c79923b78c95 service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:44.261558 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:44.267890 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 902/1818] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:33:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:33:44.783671 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a73b861-530c-46f0-bf07-4a5022f442b1 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:44.785302 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-9a73b861-530c-46f0-bf07-4a5022f442b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:44.812449 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a73b861-530c-46f0-bf07-4a5022f442b1 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:44.820212 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56189dce-5ba7-43ae-b4ca-f5cac833f2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:44.821164 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56189dce-5ba7-43ae-b4ca-f5cac833f2a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:44.821785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56189dce-5ba7-43ae-b4ca-f5cac833f2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:44.822747 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56189dce-5ba7-43ae-b4ca-f5cac833f2a6 admin admin] PATH_INFO: `/v3/projects/ce9aa7436e5d407e862975e5c8b0ef81` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:44.824917 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56189dce-5ba7-43ae-b4ca-f5cac833f2a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce9aa7436e5d407e862975e5c8b0ef81)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:44.828714 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56189dce-5ba7-43ae-b4ca-f5cac833f2a6 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:44.986378 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-56189dce-5ba7-43ae-b4ca-f5cac833f2a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce9aa7436e5d407e862975e5c8b0ef81', 'request_id': 'req-56189dce-5ba7-43ae-b4ca-f5cac833f2a6'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:44.993829 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-56189dce-5ba7-43ae-b4ca-f5cac833f2a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce9aa7436e5d407e862975e5c8b0ef81', 'request_id': 'req-56189dce-5ba7-43ae-b4ca-f5cac833f2a6'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:45.000674 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-56189dce-5ba7-43ae-b4ca-f5cac833f2a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce9aa7436e5d407e862975e5c8b0ef81', 'request_id': 'req-56189dce-5ba7-43ae-b4ca-f5cac833f2a6'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:45.120041 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-56189dce-5ba7-43ae-b4ca-f5cac833f2a6 admin admin] The token cache is being invalidate because project ce9aa7436e5d407e862975e5c8b0ef81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:33:45.120620 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-56189dce-5ba7-43ae-b4ca-f5cac833f2a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:45.124012 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-56189dce-5ba7-43ae-b4ca-f5cac833f2a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:45.133025 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 917/1819] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:33:44 2022] DELETE /identity/v3/projects/ce9aa7436e5d407e862975e5c8b0ef81 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:45.137095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9509eb3-6a1c-458b-a3ff-c79923b78c95 service neutron] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:45.164247 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9509eb3-6a1c-458b-a3ff-c79923b78c95 service neutron] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:45.308177 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a22d81e-9b19-498b-8973-ddecff46e2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:45.310668 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22d81e-9b19-498b-8973-ddecff46e2d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:45.312629 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22d81e-9b19-498b-8973-ddecff46e2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:45.313725 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22d81e-9b19-498b-8973-ddecff46e2d8 admin admin] PATH_INFO: `/v3/users/6551d84eefb84108a4fd0e208ead0a6f` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:45.317400 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a22d81e-9b19-498b-8973-ddecff46e2d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6551d84eefb84108a4fd0e208ead0a6f)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:45.320734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a22d81e-9b19-498b-8973-ddecff46e2d8 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:45.572884 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-2a22d81e-9b19-498b-8973-ddecff46e2d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6551d84eefb84108a4fd0e208ead0a6f'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:45.588527 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-2a22d81e-9b19-498b-8973-ddecff46e2d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6551d84eefb84108a4fd0e208ead0a6f'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:45.604071 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-2a22d81e-9b19-498b-8973-ddecff46e2d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6551d84eefb84108a4fd0e208ead0a6f'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:45.604919 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-2a22d81e-9b19-498b-8973-ddecff46e2d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6551d84eefb84108a4fd0e208ead0a6f'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:45.684777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-2a22d81e-9b19-498b-8973-ddecff46e2d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6551d84eefb84108a4fd0e208ead0a6f'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:45.692132 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 903/1820] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:33:45 2022] DELETE /identity/v3/users/6551d84eefb84108a4fd0e208ead0a6f => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:46.144854 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56189dce-5ba7-43ae-b4ca-f5cac833f2a6 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:46.146388 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-56189dce-5ba7-43ae-b4ca-f5cac833f2a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:46.161482 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56189dce-5ba7-43ae-b4ca-f5cac833f2a6 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:46.194133 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:46.221215 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc9d1f-5c62-4aac-8656-7ebbe482ecc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:46.222084 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc9d1f-5c62-4aac-8656-7ebbe482ecc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:46.222360 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc9d1f-5c62-4aac-8656-7ebbe482ecc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:46.222979 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc9d1f-5c62-4aac-8656-7ebbe482ecc9 admin admin] PATH_INFO: `/v3/projects/5d04d85c8a844d5382d6073e0dae5a58` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:46.224202 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdc9d1f-5c62-4aac-8656-7ebbe482ecc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d04d85c8a844d5382d6073e0dae5a58)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:46.226119 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdc9d1f-5c62-4aac-8656-7ebbe482ecc9 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:46.360275 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-6bdc9d1f-5c62-4aac-8656-7ebbe482ecc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d04d85c8a844d5382d6073e0dae5a58', 'request_id': 'req-6bdc9d1f-5c62-4aac-8656-7ebbe482ecc9'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:46.370270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-6bdc9d1f-5c62-4aac-8656-7ebbe482ecc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d04d85c8a844d5382d6073e0dae5a58', 'request_id': 'req-6bdc9d1f-5c62-4aac-8656-7ebbe482ecc9'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:46.377131 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-6bdc9d1f-5c62-4aac-8656-7ebbe482ecc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d04d85c8a844d5382d6073e0dae5a58', 'request_id': 'req-6bdc9d1f-5c62-4aac-8656-7ebbe482ecc9'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:46.445410 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-6bdc9d1f-5c62-4aac-8656-7ebbe482ecc9 admin admin] The token cache is being invalidate because project 5d04d85c8a844d5382d6073e0dae5a58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:33:46.445701 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-6bdc9d1f-5c62-4aac-8656-7ebbe482ecc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:46.446410 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-6bdc9d1f-5c62-4aac-8656-7ebbe482ecc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:46.448496 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 918/1821] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:33:46 2022] DELETE /identity/v3/projects/5d04d85c8a844d5382d6073e0dae5a58 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:46.453875 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff648c-81d2-460d-962b-cc2140d03730 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:46.454365 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff648c-81d2-460d-962b-cc2140d03730 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:46.454441 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff648c-81d2-460d-962b-cc2140d03730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:46.475370 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-c1ff648c-81d2-460d-962b-cc2140d03730 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:46.476850 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-c1ff648c-81d2-460d-962b-cc2140d03730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:46.569322 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 904/1822] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:33:46.578143 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b5ae0-9a88-4daf-a5cf-30ef24cbbff6 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:46.578858 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b5ae0-9a88-4daf-a5cf-30ef24cbbff6 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:46.579814 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b5ae0-9a88-4daf-a5cf-30ef24cbbff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:46.605238 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-cb6b5ae0-9a88-4daf-a5cf-30ef24cbbff6 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:46.607274 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-cb6b5ae0-9a88-4daf-a5cf-30ef24cbbff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:46.637683 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 919/1823] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:46.643951 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff648c-81d2-460d-962b-cc2140d03730 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:46.659537 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff648c-81d2-460d-962b-cc2140d03730 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:46.666393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3113040-ea45-427a-844d-85f72ada9974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:46.667852 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3113040-ea45-427a-844d-85f72ada9974 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:46.668911 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3113040-ea45-427a-844d-85f72ada9974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:46.669613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3113040-ea45-427a-844d-85f72ada9974 admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:46.670798 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3113040-ea45-427a-844d-85f72ada9974 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:33:46.673194 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3113040-ea45-427a-844d-85f72ada9974 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:46.673194 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3113040-ea45-427a-844d-85f72ada9974 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:46.694416 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 905/1824] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:33:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:33:46.694944 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b5ae0-9a88-4daf-a5cf-30ef24cbbff6 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:46.708233 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b5ae0-9a88-4daf-a5cf-30ef24cbbff6 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:46.716148 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cec3ae-fa69-4dfc-84fc-c36331dc1975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:46.718088 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cec3ae-fa69-4dfc-84fc-c36331dc1975 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:46.718669 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cec3ae-fa69-4dfc-84fc-c36331dc1975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:46.719226 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cec3ae-fa69-4dfc-84fc-c36331dc1975 admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:46.720448 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cec3ae-fa69-4dfc-84fc-c36331dc1975 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:46.723239 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cec3ae-fa69-4dfc-84fc-c36331dc1975 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:46.775451 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3113040-ea45-427a-844d-85f72ada9974 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:46.783346 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-f3113040-ea45-427a-844d-85f72ada9974 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:46.805604 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:46.815194 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3113040-ea45-427a-844d-85f72ada9974 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:46.822872 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 920/1825] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:33:46 2022] POST /identity/v3/projects => generated 383 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:33:46.824060 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cec3ae-fa69-4dfc-84fc-c36331dc1975 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:46.825448 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa1edd-d752-4331-8118-a7086e761043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:46.826550 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa1edd-d752-4331-8118-a7086e761043 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:46.827916 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa1edd-d752-4331-8118-a7086e761043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:46.829709 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa1edd-d752-4331-8118-a7086e761043 admin admin] PATH_INFO: `/v3/users/7b468cfe72fb427383fd161ea1826b4d` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:46.831219 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa1edd-d752-4331-8118-a7086e761043 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b468cfe72fb427383fd161ea1826b4d)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:46.835911 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa1edd-d752-4331-8118-a7086e761043 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:46.913165 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cec3ae-fa69-4dfc-84fc-c36331dc1975 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:46.918954 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618325b5-c395-4f4d-aa52-e4f610294667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:46.920484 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618325b5-c395-4f4d-aa52-e4f610294667 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:46.921788 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618325b5-c395-4f4d-aa52-e4f610294667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:46.923582 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618325b5-c395-4f4d-aa52-e4f610294667 admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:46.924340 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618325b5-c395-4f4d-aa52-e4f610294667 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:46.927183 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618325b5-c395-4f4d-aa52-e4f610294667 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:47.074647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 921/1826] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:33:46 2022] POST /identity/v3/users => generated 292 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:33:47.080507 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618325b5-c395-4f4d-aa52-e4f610294667 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:47.101983 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618325b5-c395-4f4d-aa52-e4f610294667 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:47.109240 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff9f22-63c8-4819-8e77-85bc9bc66f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:47.110218 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff9f22-63c8-4819-8e77-85bc9bc66f15 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:47.110544 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff9f22-63c8-4819-8e77-85bc9bc66f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:47.110899 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff9f22-63c8-4819-8e77-85bc9bc66f15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:47.111806 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ff9f22-63c8-4819-8e77-85bc9bc66f15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:47.113515 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ff9f22-63c8-4819-8e77-85bc9bc66f15 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:47.132456 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 922/1827] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:47 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:33:47.143807 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff9f22-63c8-4819-8e77-85bc9bc66f15 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:47.151525 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-cdfa1edd-d752-4331-8118-a7086e761043 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b468cfe72fb427383fd161ea1826b4d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:47.157480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff9f22-63c8-4819-8e77-85bc9bc66f15 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:47.164480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b189bd64-9409-43c6-9d52-71e430ffae26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:47.172198 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-cdfa1edd-d752-4331-8118-a7086e761043 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b468cfe72fb427383fd161ea1826b4d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:47.179239 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189bd64-9409-43c6-9d52-71e430ffae26 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:47.181778 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189bd64-9409-43c6-9d52-71e430ffae26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:47.182247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189bd64-9409-43c6-9d52-71e430ffae26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:47.183548 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b189bd64-9409-43c6-9d52-71e430ffae26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:47.191131 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b189bd64-9409-43c6-9d52-71e430ffae26 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:47.198416 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-cdfa1edd-d752-4331-8118-a7086e761043 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b468cfe72fb427383fd161ea1826b4d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:47.200456 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-cdfa1edd-d752-4331-8118-a7086e761043 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b468cfe72fb427383fd161ea1826b4d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:47.216641 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 923/1828] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:47 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:33:47.218339 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b189bd64-9409-43c6-9d52-71e430ffae26 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:47.224739 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:47.233975 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b189bd64-9409-43c6-9d52-71e430ffae26 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:47.241309 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e656286-4786-4cd1-b353-e7e7f7b6982c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:47.242437 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e656286-4786-4cd1-b353-e7e7f7b6982c admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:47.242873 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e656286-4786-4cd1-b353-e7e7f7b6982c admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:47.243247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e656286-4786-4cd1-b353-e7e7f7b6982c admin admin] PATH_INFO: `/v3/projects/2ca25ffa9f2b43a19a3551000c4594ef/users/6a0724b351f04577a22a143fe96e26c4/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:47.284426 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e656286-4786-4cd1-b353-e7e7f7b6982c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ca25ffa9f2b43a19a3551000c4594ef, user_id=6a0724b351f04577a22a143fe96e26c4, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:47.291554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e656286-4786-4cd1-b353-e7e7f7b6982c admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:47.309851 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-cdfa1edd-d752-4331-8118-a7086e761043 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b468cfe72fb427383fd161ea1826b4d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:47.321370 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 906/1829] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:33:46 2022] DELETE /identity/v3/users/7b468cfe72fb427383fd161ea1826b4d => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:33:47.325653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551c704-c718-4a3a-bab9-f9ebdaf2a08d None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:47.326209 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551c704-c718-4a3a-bab9-f9ebdaf2a08d None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:47.326742 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551c704-c718-4a3a-bab9-f9ebdaf2a08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:47.365553 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-d551c704-c718-4a3a-bab9-f9ebdaf2a08d None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:47.367189 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-d551c704-c718-4a3a-bab9-f9ebdaf2a08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:47.389453 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 924/1830] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:33:47 2022] PUT /identity/v3/projects/2ca25ffa9f2b43a19a3551000c4594ef/users/6a0724b351f04577a22a143fe96e26c4/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:47.391241 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec7667-fb5f-498a-a624-90b2c752c749 None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:47.391534 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec7667-fb5f-498a-a624-90b2c752c749 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:47.391842 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec7667-fb5f-498a-a624-90b2c752c749 None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:47.393166 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 925/1831] 10.222.0.25 () {54 vars in 921 bytes} [Thu Jul 7 15:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:33:47.405077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a3695-5f51-427a-a81e-154603e5f3a5 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:47.405697 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a3695-5f51-427a-a81e-154603e5f3a5 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:47.410696 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a3695-5f51-427a-a81e-154603e5f3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:47.465897 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-b48a3695-5f51-427a-a81e-154603e5f3a5 None None] MFA Rules not processed for user `6a0724b351f04577a22a143fe96e26c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:47.467184 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-b48a3695-5f51-427a-a81e-154603e5f3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:47.508999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 907/1832] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:33:47.510478 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0969ab6-f363-421e-8fc3-2207a19b0c07 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:47.511038 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0969ab6-f363-421e-8fc3-2207a19b0c07 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:47.511322 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0969ab6-f363-421e-8fc3-2207a19b0c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:47.530713 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-f0969ab6-f363-421e-8fc3-2207a19b0c07 None None] MFA Rules not processed for user `de1632555cb345f2b9213da1925c02f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:47.532222 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-f0969ab6-f363-421e-8fc3-2207a19b0c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:47.587894 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 926/1833] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:47.588670 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a3695-5f51-427a-a81e-154603e5f3a5 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:47.593438 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-b48a3695-5f51-427a-a81e-154603e5f3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:47.630077 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 908/1834] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Jul 7 15:33:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:47.631319 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0969ab6-f363-421e-8fc3-2207a19b0c07 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:47.633773 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-f0969ab6-f363-421e-8fc3-2207a19b0c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:47.647174 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:47.665775 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a3695-5f51-427a-a81e-154603e5f3a5 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:47.672170 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a5eee5-c314-4ac2-afc4-1ce44bf0319e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:47.672836 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5eee5-c314-4ac2-afc4-1ce44bf0319e service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:47.673116 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5eee5-c314-4ac2-afc4-1ce44bf0319e service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:47.673388 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5eee5-c314-4ac2-afc4-1ce44bf0319e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:47.728774 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0969ab6-f363-421e-8fc3-2207a19b0c07 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:47.734576 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8feb4e7-14e3-4683-adb6-b4b4f55dc9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:47.735238 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8feb4e7-14e3-4683-adb6-b4b4f55dc9dc admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:47.735550 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8feb4e7-14e3-4683-adb6-b4b4f55dc9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:47.735864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8feb4e7-14e3-4683-adb6-b4b4f55dc9dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:47.736557 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8feb4e7-14e3-4683-adb6-b4b4f55dc9dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:47.738378 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a5eee5-c314-4ac2-afc4-1ce44bf0319e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:47.738764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8feb4e7-14e3-4683-adb6-b4b4f55dc9dc admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:47.739912 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a5eee5-c314-4ac2-afc4-1ce44bf0319e service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:47.758656 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 927/1835] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:33:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:33:47.758656 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a5eee5-c314-4ac2-afc4-1ce44bf0319e service neutron] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:47.760195 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-95a5eee5-c314-4ac2-afc4-1ce44bf0319e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:47.787308 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a5eee5-c314-4ac2-afc4-1ce44bf0319e service neutron] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:47.790377 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a4b10-6cf2-4830-8b58-513130f4f152 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:47.791132 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a4b10-6cf2-4830-8b58-513130f4f152 service nova] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:47.791426 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a4b10-6cf2-4830-8b58-513130f4f152 service nova] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:47.791771 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a4b10-6cf2-4830-8b58-513130f4f152 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:47.801878 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822a4b10-6cf2-4830-8b58-513130f4f152 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:47.803643 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822a4b10-6cf2-4830-8b58-513130f4f152 service nova] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:47.814460 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 928/1836] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:33:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:33:47.866489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 909/1837] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:33:47 2022] POST /identity/v3/projects => generated 385 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:33:47.870506 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a4b10-6cf2-4830-8b58-513130f4f152 service nova] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:47.880952 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a4b10-6cf2-4830-8b58-513130f4f152 service nova] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:47.936710 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa88e4e-5954-464f-9292-918e712fecc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:47.937348 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa88e4e-5954-464f-9292-918e712fecc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:47.937629 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa88e4e-5954-464f-9292-918e712fecc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:47.937899 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa88e4e-5954-464f-9292-918e712fecc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:47.938579 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa88e4e-5954-464f-9292-918e712fecc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:47.940521 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa88e4e-5954-464f-9292-918e712fecc5 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:48.053840 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 929/1838] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:33:47 2022] POST /identity/v3/users => generated 299 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:33:48.057693 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8feb4e7-14e3-4683-adb6-b4b4f55dc9dc admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:48.062171 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:48.069463 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8feb4e7-14e3-4683-adb6-b4b4f55dc9dc admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:48.074132 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e399a-d98f-4739-94ff-c63109fc9039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:48.074728 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e399a-d98f-4739-94ff-c63109fc9039 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:48.075007 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e399a-d98f-4739-94ff-c63109fc9039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:48.075292 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e399a-d98f-4739-94ff-c63109fc9039 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:48.076046 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9e399a-d98f-4739-94ff-c63109fc9039 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:48.077827 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9e399a-d98f-4739-94ff-c63109fc9039 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:48.089451 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 910/1839] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:33:48.093949 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa88e4e-5954-464f-9292-918e712fecc5 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:48.104067 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa88e4e-5954-464f-9292-918e712fecc5 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:48.109730 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afdb042-9ed2-4545-9b9e-918101c413d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:48.110618 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afdb042-9ed2-4545-9b9e-918101c413d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:48.110904 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afdb042-9ed2-4545-9b9e-918101c413d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:48.111182 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afdb042-9ed2-4545-9b9e-918101c413d5 admin admin] PATH_INFO: `/v3/projects/252d5b8ce105419cbcc31870cd0e0e96/users/d1d11bd791734fc4a457321d180e1c79/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:48.126244 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afdb042-9ed2-4545-9b9e-918101c413d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=252d5b8ce105419cbcc31870cd0e0e96, user_id=d1d11bd791734fc4a457321d180e1c79, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:48.128991 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afdb042-9ed2-4545-9b9e-918101c413d5 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:48.174945 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e399a-d98f-4739-94ff-c63109fc9039 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:48.176896 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-6d9e399a-d98f-4739-94ff-c63109fc9039 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:48.193063 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e399a-d98f-4739-94ff-c63109fc9039 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:48.196946 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 930/1840] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:33:48 2022] PUT /identity/v3/projects/252d5b8ce105419cbcc31870cd0e0e96/users/d1d11bd791734fc4a457321d180e1c79/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:48.214197 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afdb042-9ed2-4545-9b9e-918101c413d5 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:48.271546 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0db805b-7121-4971-8153-30594e1de610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:48.272482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db805b-7121-4971-8153-30594e1de610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:48.276622 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db805b-7121-4971-8153-30594e1de610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:48.276622 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db805b-7121-4971-8153-30594e1de610 admin admin] PATH_INFO: `/v3/projects/6c773c85cddf49a99b435c9534205e44` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:48.276622 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0db805b-7121-4971-8153-30594e1de610 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c773c85cddf49a99b435c9534205e44)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:48.276622 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0db805b-7121-4971-8153-30594e1de610 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:48.284057 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afdb042-9ed2-4545-9b9e-918101c413d5 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:48.288720 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1a26c-30be-41ab-bfcb-729268b1c103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:48.289455 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1a26c-30be-41ab-bfcb-729268b1c103 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:48.289806 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1a26c-30be-41ab-bfcb-729268b1c103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:48.290154 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1a26c-30be-41ab-bfcb-729268b1c103 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:48.290888 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee1a26c-30be-41ab-bfcb-729268b1c103 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:48.292524 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee1a26c-30be-41ab-bfcb-729268b1c103 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:48.303291 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 931/1841] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:33:48.307404 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1a26c-30be-41ab-bfcb-729268b1c103 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:48.317335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1a26c-30be-41ab-bfcb-729268b1c103 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:48.321857 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c581d82-4dc6-41e9-968d-42b3219d6379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:48.322694 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c581d82-4dc6-41e9-968d-42b3219d6379 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:48.322976 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c581d82-4dc6-41e9-968d-42b3219d6379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:48.323249 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c581d82-4dc6-41e9-968d-42b3219d6379 admin admin] PATH_INFO: `/v3/projects/252d5b8ce105419cbcc31870cd0e0e96/users/d1d11bd791734fc4a457321d180e1c79/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:48.325284 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c581d82-4dc6-41e9-968d-42b3219d6379 admin admin] RBAC: Authorizing `identity:create_grant(project_id=252d5b8ce105419cbcc31870cd0e0e96, user_id=d1d11bd791734fc4a457321d180e1c79, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:48.327908 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c581d82-4dc6-41e9-968d-42b3219d6379 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:48.337822 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 932/1842] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:33:48 2022] PUT /identity/v3/projects/252d5b8ce105419cbcc31870cd0e0e96/users/d1d11bd791734fc4a457321d180e1c79/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:33:48.348848 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12056bb7-0a47-403f-a27b-12a395cc54a1 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:48.349138 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12056bb7-0a47-403f-a27b-12a395cc54a1 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:48.349414 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12056bb7-0a47-403f-a27b-12a395cc54a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:48.376297 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-12056bb7-0a47-403f-a27b-12a395cc54a1 None None] MFA Rules not processed for user `d1d11bd791734fc4a457321d180e1c79`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:48.377496 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-12056bb7-0a47-403f-a27b-12a395cc54a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:48.417210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-e0db805b-7121-4971-8153-30594e1de610 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c773c85cddf49a99b435c9534205e44', 'request_id': 'req-e0db805b-7121-4971-8153-30594e1de610'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:48.427025 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-e0db805b-7121-4971-8153-30594e1de610 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c773c85cddf49a99b435c9534205e44', 'request_id': 'req-e0db805b-7121-4971-8153-30594e1de610'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:48.428954 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:48.436653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-e0db805b-7121-4971-8153-30594e1de610 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c773c85cddf49a99b435c9534205e44', 'request_id': 'req-e0db805b-7121-4971-8153-30594e1de610'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:48.460560 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 933/1843] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:48 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:33:48.472170 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12056bb7-0a47-403f-a27b-12a395cc54a1 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:48.473574 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-12056bb7-0a47-403f-a27b-12a395cc54a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:48.532975 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12056bb7-0a47-403f-a27b-12a395cc54a1 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:48.573587 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-e0db805b-7121-4971-8153-30594e1de610 admin admin] The token cache is being invalidate because project 6c773c85cddf49a99b435c9534205e44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:33:48.574493 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-e0db805b-7121-4971-8153-30594e1de610 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:48.575828 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-e0db805b-7121-4971-8153-30594e1de610 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:48.580863 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 911/1844] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:33:48 2022] DELETE /identity/v3/projects/6c773c85cddf49a99b435c9534205e44 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:33:48.582012 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0db805b-7121-4971-8153-30594e1de610 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:48.583495 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-e0db805b-7121-4971-8153-30594e1de610 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:48.593966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f346975e-3c97-4ba9-a8b5-514616a897d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:48.597098 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346975e-3c97-4ba9-a8b5-514616a897d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:48.597098 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346975e-3c97-4ba9-a8b5-514616a897d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:48.597098 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346975e-3c97-4ba9-a8b5-514616a897d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:48.597350 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-f346975e-3c97-4ba9-a8b5-514616a897d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:48.599865 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:48.671319 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f346975e-3c97-4ba9-a8b5-514616a897d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:48.672600 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f346975e-3c97-4ba9-a8b5-514616a897d0 service cinder] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:48.673070 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0db805b-7121-4971-8153-30594e1de610 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:48.682238 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 934/1845] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:33:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 15:33:48.688785 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c7234-c3bc-4e24-ba6c-21cdba7e6bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:48.689531 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c7234-c3bc-4e24-ba6c-21cdba7e6bdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:48.689784 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c7234-c3bc-4e24-ba6c-21cdba7e6bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:48.690133 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c7234-c3bc-4e24-ba6c-21cdba7e6bdf admin admin] PATH_INFO: `/v3/users/9fe991dc33f3440498f895fca6ba8f1c` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:48.691499 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692c7234-c3bc-4e24-ba6c-21cdba7e6bdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fe991dc33f3440498f895fca6ba8f1c)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:48.693603 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692c7234-c3bc-4e24-ba6c-21cdba7e6bdf admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:48.915504 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-692c7234-c3bc-4e24-ba6c-21cdba7e6bdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fe991dc33f3440498f895fca6ba8f1c'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:48.922751 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-692c7234-c3bc-4e24-ba6c-21cdba7e6bdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fe991dc33f3440498f895fca6ba8f1c'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:48.934611 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-692c7234-c3bc-4e24-ba6c-21cdba7e6bdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fe991dc33f3440498f895fca6ba8f1c'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:48.935304 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-692c7234-c3bc-4e24-ba6c-21cdba7e6bdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fe991dc33f3440498f895fca6ba8f1c'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:48.995704 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-692c7234-c3bc-4e24-ba6c-21cdba7e6bdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fe991dc33f3440498f895fca6ba8f1c'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:49.002737 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 912/1846] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:33:48 2022] DELETE /identity/v3/users/9fe991dc33f3440498f895fca6ba8f1c => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:49.431535 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f346975e-3c97-4ba9-a8b5-514616a897d0 service cinder] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:49.445735 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f346975e-3c97-4ba9-a8b5-514616a897d0 service cinder] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:49.505253 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:49.518094 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5bdc1a-3057-4b82-b1c4-85b2ccbb00b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:49.518991 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5bdc1a-3057-4b82-b1c4-85b2ccbb00b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:49.519236 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5bdc1a-3057-4b82-b1c4-85b2ccbb00b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:49.519514 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5bdc1a-3057-4b82-b1c4-85b2ccbb00b4 admin admin] PATH_INFO: `/v3/projects/454dc6a84056496b9e26c6adf9000cfc` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:49.525341 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5bdc1a-3057-4b82-b1c4-85b2ccbb00b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=454dc6a84056496b9e26c6adf9000cfc)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:49.526779 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5bdc1a-3057-4b82-b1c4-85b2ccbb00b4 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:49.688046 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-8a5bdc1a-3057-4b82-b1c4-85b2ccbb00b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '454dc6a84056496b9e26c6adf9000cfc', 'request_id': 'req-8a5bdc1a-3057-4b82-b1c4-85b2ccbb00b4'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:49.693829 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-8a5bdc1a-3057-4b82-b1c4-85b2ccbb00b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '454dc6a84056496b9e26c6adf9000cfc', 'request_id': 'req-8a5bdc1a-3057-4b82-b1c4-85b2ccbb00b4'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:49.700581 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-8a5bdc1a-3057-4b82-b1c4-85b2ccbb00b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '454dc6a84056496b9e26c6adf9000cfc', 'request_id': 'req-8a5bdc1a-3057-4b82-b1c4-85b2ccbb00b4'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:49.775528 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-8a5bdc1a-3057-4b82-b1c4-85b2ccbb00b4 admin admin] The token cache is being invalidate because project 454dc6a84056496b9e26c6adf9000cfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:33:49.775929 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-8a5bdc1a-3057-4b82-b1c4-85b2ccbb00b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:49.776501 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-8a5bdc1a-3057-4b82-b1c4-85b2ccbb00b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:49.780626 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 935/1847] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:33:49 2022] DELETE /identity/v3/projects/454dc6a84056496b9e26c6adf9000cfc => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:33:49.784433 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f2213-cc05-40bc-bc3b-5027789f31f7 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:49.785198 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f2213-cc05-40bc-bc3b-5027789f31f7 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:49.785483 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f2213-cc05-40bc-bc3b-5027789f31f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:49.807787 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-352f2213-cc05-40bc-bc3b-5027789f31f7 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:49.809053 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-352f2213-cc05-40bc-bc3b-5027789f31f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:49.879324 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 913/1848] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:49.888198 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d452944-1ba6-48f1-b9de-380ab1e3cb5e None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:49.888493 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d452944-1ba6-48f1-b9de-380ab1e3cb5e None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:49.888804 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d452944-1ba6-48f1-b9de-380ab1e3cb5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:49.907539 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-3d452944-1ba6-48f1-b9de-380ab1e3cb5e None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:49.908801 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-3d452944-1ba6-48f1-b9de-380ab1e3cb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:49.919399 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 936/1849] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:49.926110 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352f2213-cc05-40bc-bc3b-5027789f31f7 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:49.938063 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352f2213-cc05-40bc-bc3b-5027789f31f7 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:49.943288 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d62642-4752-42c4-bfef-ffcc2f06fe27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:49.944201 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d62642-4752-42c4-bfef-ffcc2f06fe27 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:49.944471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d62642-4752-42c4-bfef-ffcc2f06fe27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:49.945987 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d62642-4752-42c4-bfef-ffcc2f06fe27 admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:49.945987 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d62642-4752-42c4-bfef-ffcc2f06fe27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:33:49.945987 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d62642-4752-42c4-bfef-ffcc2f06fe27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:49.947620 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d62642-4752-42c4-bfef-ffcc2f06fe27 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:49.969379 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 914/1850] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:33:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:33:49.973591 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d452944-1ba6-48f1-b9de-380ab1e3cb5e None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:49.983892 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d452944-1ba6-48f1-b9de-380ab1e3cb5e None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:49.988649 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6494ef3-fc3e-42f1-be43-a64472816ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:49.989379 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6494ef3-fc3e-42f1-be43-a64472816ffb admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:49.989730 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6494ef3-fc3e-42f1-be43-a64472816ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:49.990072 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6494ef3-fc3e-42f1-be43-a64472816ffb admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:49.990826 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6494ef3-fc3e-42f1-be43-a64472816ffb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:49.992759 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6494ef3-fc3e-42f1-be43-a64472816ffb admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:50.089915 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 937/1851] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:33:49 2022] POST /identity/v3/projects => generated 369 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:33:50.091044 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d62642-4752-42c4-bfef-ffcc2f06fe27 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:50.101397 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d62642-4752-42c4-bfef-ffcc2f06fe27 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:50.119724 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:50.161879 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db55f0b3-4f9a-421f-84dd-30f01a451924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:50.162670 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55f0b3-4f9a-421f-84dd-30f01a451924 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:50.162955 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55f0b3-4f9a-421f-84dd-30f01a451924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:50.163223 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55f0b3-4f9a-421f-84dd-30f01a451924 admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:50.164012 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db55f0b3-4f9a-421f-84dd-30f01a451924 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:50.165878 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db55f0b3-4f9a-421f-84dd-30f01a451924 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:50.250465 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 915/1852] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:33:50 2022] POST /identity/v3/users => generated 285 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:33:50.254464 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6494ef3-fc3e-42f1-be43-a64472816ffb admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:50.264671 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6494ef3-fc3e-42f1-be43-a64472816ffb admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:50.269145 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29e625-dfa9-481f-bf4e-07e78a83c465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:50.274388 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29e625-dfa9-481f-bf4e-07e78a83c465 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:50.274388 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29e625-dfa9-481f-bf4e-07e78a83c465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:50.274388 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29e625-dfa9-481f-bf4e-07e78a83c465 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:50.274388 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29e625-dfa9-481f-bf4e-07e78a83c465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:50.274388 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29e625-dfa9-481f-bf4e-07e78a83c465 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:50.283249 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 938/1853] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:33:50.287346 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db55f0b3-4f9a-421f-84dd-30f01a451924 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:50.297937 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db55f0b3-4f9a-421f-84dd-30f01a451924 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:50.302469 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1ed7f-fbfe-449d-8e33-eeccf902031b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:50.303117 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1ed7f-fbfe-449d-8e33-eeccf902031b admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:50.303394 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1ed7f-fbfe-449d-8e33-eeccf902031b admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:50.303727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1ed7f-fbfe-449d-8e33-eeccf902031b admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:50.304446 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec1ed7f-fbfe-449d-8e33-eeccf902031b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:50.306004 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec1ed7f-fbfe-449d-8e33-eeccf902031b admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:50.316810 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 916/1854] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:33:50.320738 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29e625-dfa9-481f-bf4e-07e78a83c465 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:50.330532 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29e625-dfa9-481f-bf4e-07e78a83c465 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:50.335230 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72498c05-01fc-4e31-92ba-c9ccdc771615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:50.336136 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72498c05-01fc-4e31-92ba-c9ccdc771615 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:50.336420 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72498c05-01fc-4e31-92ba-c9ccdc771615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:50.336696 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72498c05-01fc-4e31-92ba-c9ccdc771615 admin admin] PATH_INFO: `/v3/projects/cc960c94668e4aa687c2b692791ec4c8/users/09441d3ca1ea41d98c9cb1ad69665a2a/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:50.350792 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72498c05-01fc-4e31-92ba-c9ccdc771615 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc960c94668e4aa687c2b692791ec4c8, user_id=09441d3ca1ea41d98c9cb1ad69665a2a, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:50.353457 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72498c05-01fc-4e31-92ba-c9ccdc771615 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:50.432328 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 939/1855] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:33:50 2022] PUT /identity/v3/projects/cc960c94668e4aa687c2b692791ec4c8/users/09441d3ca1ea41d98c9cb1ad69665a2a/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:50.443722 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04baedc-a3c1-42a0-9712-badc40485408 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:50.444066 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04baedc-a3c1-42a0-9712-badc40485408 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:50.444350 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04baedc-a3c1-42a0-9712-badc40485408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:50.471093 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-a04baedc-a3c1-42a0-9712-badc40485408 None None] MFA Rules not processed for user `09441d3ca1ea41d98c9cb1ad69665a2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:50.472349 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-a04baedc-a3c1-42a0-9712-badc40485408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:50.535648 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 917/1856] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:50 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:50.540083 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72498c05-01fc-4e31-92ba-c9ccdc771615 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:50.611500 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72498c05-01fc-4e31-92ba-c9ccdc771615 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:50.616134 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8015a9c-ae43-4fda-a53a-dab5e0b8b9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:50.616837 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8015a9c-ae43-4fda-a53a-dab5e0b8b9dc admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:50.617118 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8015a9c-ae43-4fda-a53a-dab5e0b8b9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:50.617393 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8015a9c-ae43-4fda-a53a-dab5e0b8b9dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:50.618286 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8015a9c-ae43-4fda-a53a-dab5e0b8b9dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:50.621111 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8015a9c-ae43-4fda-a53a-dab5e0b8b9dc admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:50.699118 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:50.705743 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 940/1857] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:33:50 2022] POST /identity/v3/projects => generated 365 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:33:50.709881 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04baedc-a3c1-42a0-9712-badc40485408 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:50.721192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04baedc-a3c1-42a0-9712-badc40485408 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:50.776750 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3db3ad-8b9a-4f7f-8f31-55e312d3a809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:50.777407 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3db3ad-8b9a-4f7f-8f31-55e312d3a809 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:50.777691 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3db3ad-8b9a-4f7f-8f31-55e312d3a809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:50.777972 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3db3ad-8b9a-4f7f-8f31-55e312d3a809 admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:50.778652 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3db3ad-8b9a-4f7f-8f31-55e312d3a809 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:50.781127 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3db3ad-8b9a-4f7f-8f31-55e312d3a809 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:50.864055 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 918/1858] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:33:50 2022] POST /identity/v3/users => generated 289 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:33:50.865198 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8015a9c-ae43-4fda-a53a-dab5e0b8b9dc admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:50.876029 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8015a9c-ae43-4fda-a53a-dab5e0b8b9dc admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:50.889822 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c29ad0-9412-4eb1-b8e9-fe382f0177c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:50.890524 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c29ad0-9412-4eb1-b8e9-fe382f0177c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:50.890825 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c29ad0-9412-4eb1-b8e9-fe382f0177c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:50.891091 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c29ad0-9412-4eb1-b8e9-fe382f0177c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:50.891874 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c29ad0-9412-4eb1-b8e9-fe382f0177c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:50.893490 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c29ad0-9412-4eb1-b8e9-fe382f0177c7 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:50.908597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 941/1859] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:50 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:33:50.912664 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3db3ad-8b9a-4f7f-8f31-55e312d3a809 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:50.924493 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3db3ad-8b9a-4f7f-8f31-55e312d3a809 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:50.931280 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd08fd1-653e-4e54-b4a5-26509700f13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:50.931455 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd08fd1-653e-4e54-b4a5-26509700f13e admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:50.931772 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd08fd1-653e-4e54-b4a5-26509700f13e admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:50.932105 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd08fd1-653e-4e54-b4a5-26509700f13e admin admin] PATH_INFO: `/v3/projects/e989357ca6a74fe1a93f986d3adb98ea/users/da9660bf705b4e86b734a380cd49ee19/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:50.960504 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd08fd1-653e-4e54-b4a5-26509700f13e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e989357ca6a74fe1a93f986d3adb98ea, user_id=da9660bf705b4e86b734a380cd49ee19, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:50.970517 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd08fd1-653e-4e54-b4a5-26509700f13e admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:50.975453 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:51.051250 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 919/1860] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:33:50 2022] PUT /identity/v3/projects/e989357ca6a74fe1a93f986d3adb98ea/users/da9660bf705b4e86b734a380cd49ee19/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:51.052818 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c29ad0-9412-4eb1-b8e9-fe382f0177c7 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:51.062606 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c29ad0-9412-4eb1-b8e9-fe382f0177c7 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:51.131141 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8198c311-e053-4c43-8a81-759faa5ddd38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:51.132336 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198c311-e053-4c43-8a81-759faa5ddd38 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:51.133125 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198c311-e053-4c43-8a81-759faa5ddd38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:51.133410 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198c311-e053-4c43-8a81-759faa5ddd38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:51.134295 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8198c311-e053-4c43-8a81-759faa5ddd38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:51.135904 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8198c311-e053-4c43-8a81-759faa5ddd38 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:51.147652 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 942/1861] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:33:51.151697 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd08fd1-653e-4e54-b4a5-26509700f13e admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:51.167266 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd08fd1-653e-4e54-b4a5-26509700f13e admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:51.173892 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40edf52-fe26-4ef2-8926-1a42e6b08082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:51.175203 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40edf52-fe26-4ef2-8926-1a42e6b08082 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:51.175763 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40edf52-fe26-4ef2-8926-1a42e6b08082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:51.176537 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40edf52-fe26-4ef2-8926-1a42e6b08082 admin admin] PATH_INFO: `/v3/projects/e989357ca6a74fe1a93f986d3adb98ea/users/da9660bf705b4e86b734a380cd49ee19/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:51.180021 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40edf52-fe26-4ef2-8926-1a42e6b08082 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e989357ca6a74fe1a93f986d3adb98ea, user_id=da9660bf705b4e86b734a380cd49ee19, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:51.183163 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40edf52-fe26-4ef2-8926-1a42e6b08082 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:51.197628 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 920/1862] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:33:51 2022] PUT /identity/v3/projects/e989357ca6a74fe1a93f986d3adb98ea/users/da9660bf705b4e86b734a380cd49ee19/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:51.212196 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312ce1c-3a77-4872-8455-eca3c0bb2904 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:51.212196 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312ce1c-3a77-4872-8455-eca3c0bb2904 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:51.212196 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312ce1c-3a77-4872-8455-eca3c0bb2904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:51.247363 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-0312ce1c-3a77-4872-8455-eca3c0bb2904 None None] MFA Rules not processed for user `da9660bf705b4e86b734a380cd49ee19`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:51.248734 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-0312ce1c-3a77-4872-8455-eca3c0bb2904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:51.334992 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 943/1863] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:51 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:51.342431 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40edf52-fe26-4ef2-8926-1a42e6b08082 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:51.344129 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-f40edf52-fe26-4ef2-8926-1a42e6b08082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:51.404556 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40edf52-fe26-4ef2-8926-1a42e6b08082 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:51.409821 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4f30d-68b4-4652-8bca-d077b1344713 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:51.410895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4f30d-68b4-4652-8bca-d077b1344713 service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:51.411192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4f30d-68b4-4652-8bca-d077b1344713 service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:51.411476 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4f30d-68b4-4652-8bca-d077b1344713 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:51.422108 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e4f30d-68b4-4652-8bca-d077b1344713 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:51.423549 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e4f30d-68b4-4652-8bca-d077b1344713 service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:51.433596 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 921/1864] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:33:51 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:33:55.478564 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312ce1c-3a77-4872-8455-eca3c0bb2904 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:55.488223 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312ce1c-3a77-4872-8455-eca3c0bb2904 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:55.492079 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0831bc-9ca7-473f-bdba-a98fd273bf93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:55.492768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0831bc-9ca7-473f-bdba-a98fd273bf93 service cinder] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:55.493078 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0831bc-9ca7-473f-bdba-a98fd273bf93 service cinder] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:55.501949 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0831bc-9ca7-473f-bdba-a98fd273bf93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:55.501949 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-ee0831bc-9ca7-473f-bdba-a98fd273bf93 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:55.558957 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0831bc-9ca7-473f-bdba-a98fd273bf93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:55.560463 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0831bc-9ca7-473f-bdba-a98fd273bf93 service cinder] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:55.570313 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 944/1865] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:33:55 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:33:55.776390 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4f30d-68b4-4652-8bca-d077b1344713 service cinder] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:55.846503 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4f30d-68b4-4652-8bca-d077b1344713 service cinder] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:55.851142 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53adb77a-e35e-4cd4-baac-2997e6591e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:55.852060 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53adb77a-e35e-4cd4-baac-2997e6591e60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:55.852347 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53adb77a-e35e-4cd4-baac-2997e6591e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:55.852620 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53adb77a-e35e-4cd4-baac-2997e6591e60 admin admin] PATH_INFO: `/v3/users/09441d3ca1ea41d98c9cb1ad69665a2a` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:55.854063 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53adb77a-e35e-4cd4-baac-2997e6591e60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09441d3ca1ea41d98c9cb1ad69665a2a)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:55.856286 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53adb77a-e35e-4cd4-baac-2997e6591e60 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:56.109437 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-53adb77a-e35e-4cd4-baac-2997e6591e60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09441d3ca1ea41d98c9cb1ad69665a2a'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:56.116518 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-53adb77a-e35e-4cd4-baac-2997e6591e60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09441d3ca1ea41d98c9cb1ad69665a2a'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:56.149197 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-53adb77a-e35e-4cd4-baac-2997e6591e60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09441d3ca1ea41d98c9cb1ad69665a2a'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:56.150248 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-53adb77a-e35e-4cd4-baac-2997e6591e60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09441d3ca1ea41d98c9cb1ad69665a2a'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:56.226355 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-53adb77a-e35e-4cd4-baac-2997e6591e60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09441d3ca1ea41d98c9cb1ad69665a2a'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:56.233686 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 922/1866] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:33:55 2022] DELETE /identity/v3/users/09441d3ca1ea41d98c9cb1ad69665a2a => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:56.238177 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6743c9-330a-4ddc-88fc-c437183f67d6 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:56.238562 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6743c9-330a-4ddc-88fc-c437183f67d6 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:56.238844 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6743c9-330a-4ddc-88fc-c437183f67d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:56.246878 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:56.258478 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-ac6743c9-330a-4ddc-88fc-c437183f67d6 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:56.259672 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-ac6743c9-330a-4ddc-88fc-c437183f67d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:56.326862 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53adb77a-e35e-4cd4-baac-2997e6591e60 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:56.327086 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-53adb77a-e35e-4cd4-baac-2997e6591e60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:56.327805 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 945/1867] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:56.338163 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6743c9-330a-4ddc-88fc-c437183f67d6 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:56.344532 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-ac6743c9-330a-4ddc-88fc-c437183f67d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:56.349871 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:56.360381 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53adb77a-e35e-4cd4-baac-2997e6591e60 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:56.370623 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83adb851-4f35-4fb2-9a43-480a832cd62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:56.372256 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83adb851-4f35-4fb2-9a43-480a832cd62c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:56.372655 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83adb851-4f35-4fb2-9a43-480a832cd62c admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:56.373630 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83adb851-4f35-4fb2-9a43-480a832cd62c admin admin] PATH_INFO: `/v3/users/1fb7b53dac284a62b694e0e6af0a7a89` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:56.375599 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83adb851-4f35-4fb2-9a43-480a832cd62c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fb7b53dac284a62b694e0e6af0a7a89)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:56.378042 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83adb851-4f35-4fb2-9a43-480a832cd62c admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:56.415733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6743c9-330a-4ddc-88fc-c437183f67d6 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:56.420612 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a8bdbf-f50b-4102-92e7-67b4608950bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:56.423776 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8bdbf-f50b-4102-92e7-67b4608950bd service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:56.423776 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8bdbf-f50b-4102-92e7-67b4608950bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:56.423776 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8bdbf-f50b-4102-92e7-67b4608950bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:56.437958 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a8bdbf-f50b-4102-92e7-67b4608950bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:56.439502 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a8bdbf-f50b-4102-92e7-67b4608950bd service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:56.450028 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 946/1868] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:33:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:33:56.569619 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-83adb851-4f35-4fb2-9a43-480a832cd62c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fb7b53dac284a62b694e0e6af0a7a89'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:56.576431 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-83adb851-4f35-4fb2-9a43-480a832cd62c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fb7b53dac284a62b694e0e6af0a7a89'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:56.589058 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-83adb851-4f35-4fb2-9a43-480a832cd62c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fb7b53dac284a62b694e0e6af0a7a89'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:56.589773 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-83adb851-4f35-4fb2-9a43-480a832cd62c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fb7b53dac284a62b694e0e6af0a7a89'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:56.669403 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-83adb851-4f35-4fb2-9a43-480a832cd62c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fb7b53dac284a62b694e0e6af0a7a89'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:56.676983 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 923/1869] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:33:56 2022] DELETE /identity/v3/users/1fb7b53dac284a62b694e0e6af0a7a89 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:56.681414 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ad976-1cef-4186-bae8-f81aa154b7ee None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:56.681905 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ad976-1cef-4186-bae8-f81aa154b7ee None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:56.682009 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ad976-1cef-4186-bae8-f81aa154b7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:56.699932 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-b09ad976-1cef-4186-bae8-f81aa154b7ee None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:56.701158 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-b09ad976-1cef-4186-bae8-f81aa154b7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:56.763973 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 947/1870] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:56.774314 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83adb851-4f35-4fb2-9a43-480a832cd62c admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:56.775898 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-83adb851-4f35-4fb2-9a43-480a832cd62c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:56.837219 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83adb851-4f35-4fb2-9a43-480a832cd62c admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:56.841111 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493b1b7-534d-475b-bf80-260f2000937a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:56.842275 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493b1b7-534d-475b-bf80-260f2000937a service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:56.842561 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493b1b7-534d-475b-bf80-260f2000937a service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:56.842844 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493b1b7-534d-475b-bf80-260f2000937a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:56.852317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6493b1b7-534d-475b-bf80-260f2000937a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:56.854063 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6493b1b7-534d-475b-bf80-260f2000937a service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:56.854191 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ad976-1cef-4186-bae8-f81aa154b7ee None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:56.855756 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-b09ad976-1cef-4186-bae8-f81aa154b7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:56.871315 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ad976-1cef-4186-bae8-f81aa154b7ee None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:56.872101 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 924/1871] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:33:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:33:56.874471 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cdca3a-2faa-4ddd-bced-3c1052d12ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:56.875379 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cdca3a-2faa-4ddd-bced-3c1052d12ace admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:56.875701 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cdca3a-2faa-4ddd-bced-3c1052d12ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:56.875980 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cdca3a-2faa-4ddd-bced-3c1052d12ace admin admin] PATH_INFO: `/v3/projects/cc960c94668e4aa687c2b692791ec4c8` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:56.877410 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cdca3a-2faa-4ddd-bced-3c1052d12ace admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc960c94668e4aa687c2b692791ec4c8)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:56.879315 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cdca3a-2faa-4ddd-bced-3c1052d12ace admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:56.983484 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:56.987371 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-a1cdca3a-2faa-4ddd-bced-3c1052d12ace admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc960c94668e4aa687c2b692791ec4c8', 'request_id': 'req-a1cdca3a-2faa-4ddd-bced-3c1052d12ace'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:56.993249 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-a1cdca3a-2faa-4ddd-bced-3c1052d12ace admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc960c94668e4aa687c2b692791ec4c8', 'request_id': 'req-a1cdca3a-2faa-4ddd-bced-3c1052d12ace'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:56.999443 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-a1cdca3a-2faa-4ddd-bced-3c1052d12ace admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc960c94668e4aa687c2b692791ec4c8', 'request_id': 'req-a1cdca3a-2faa-4ddd-bced-3c1052d12ace'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:57.076015 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-a1cdca3a-2faa-4ddd-bced-3c1052d12ace admin admin] The token cache is being invalidate because project cc960c94668e4aa687c2b692791ec4c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:33:57.076933 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-a1cdca3a-2faa-4ddd-bced-3c1052d12ace admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:57.077823 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-a1cdca3a-2faa-4ddd-bced-3c1052d12ace admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:57.082572 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 948/1872] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:33:56 2022] DELETE /identity/v3/projects/cc960c94668e4aa687c2b692791ec4c8 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:57.083739 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493b1b7-534d-475b-bf80-260f2000937a service neutron] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:57.094967 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493b1b7-534d-475b-bf80-260f2000937a service neutron] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:57.153413 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2662c-d68e-4de8-a32b-7e55d3fc0c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:57.154852 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2662c-d68e-4de8-a32b-7e55d3fc0c82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:57.155247 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2662c-d68e-4de8-a32b-7e55d3fc0c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:57.155526 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2662c-d68e-4de8-a32b-7e55d3fc0c82 admin admin] PATH_INFO: `/v3/users/da9660bf705b4e86b734a380cd49ee19` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:57.157226 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf2662c-d68e-4de8-a32b-7e55d3fc0c82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da9660bf705b4e86b734a380cd49ee19)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:57.159131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf2662c-d68e-4de8-a32b-7e55d3fc0c82 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:57.180533 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:57.309030 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cdca3a-2faa-4ddd-bced-3c1052d12ace admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:57.311143 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-a1cdca3a-2faa-4ddd-bced-3c1052d12ace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:57.335612 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cdca3a-2faa-4ddd-bced-3c1052d12ace admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:57.340306 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9dfc80-27a4-4b41-b70d-c3198e038acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:57.341271 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9dfc80-27a4-4b41-b70d-c3198e038acc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:57.341623 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9dfc80-27a4-4b41-b70d-c3198e038acc admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:57.342095 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9dfc80-27a4-4b41-b70d-c3198e038acc admin admin] PATH_INFO: `/v3/projects/12a7c17b37ab40999cf11094f358f8ed` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:57.343366 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9dfc80-27a4-4b41-b70d-c3198e038acc admin admin] RBAC: Authorizing `identity:delete_project(project_id=12a7c17b37ab40999cf11094f358f8ed)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:57.344698 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-4bf2662c-d68e-4de8-a32b-7e55d3fc0c82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da9660bf705b4e86b734a380cd49ee19'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:57.345595 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9dfc80-27a4-4b41-b70d-c3198e038acc admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:57.363858 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-4bf2662c-d68e-4de8-a32b-7e55d3fc0c82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da9660bf705b4e86b734a380cd49ee19'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:57.377603 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-4bf2662c-d68e-4de8-a32b-7e55d3fc0c82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da9660bf705b4e86b734a380cd49ee19'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:57.378321 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-4bf2662c-d68e-4de8-a32b-7e55d3fc0c82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da9660bf705b4e86b734a380cd49ee19'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:57.455523 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-9d9dfc80-27a4-4b41-b70d-c3198e038acc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12a7c17b37ab40999cf11094f358f8ed', 'request_id': 'req-9d9dfc80-27a4-4b41-b70d-c3198e038acc'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:57.460955 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-9d9dfc80-27a4-4b41-b70d-c3198e038acc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12a7c17b37ab40999cf11094f358f8ed', 'request_id': 'req-9d9dfc80-27a4-4b41-b70d-c3198e038acc'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:57.466351 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-4bf2662c-d68e-4de8-a32b-7e55d3fc0c82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da9660bf705b4e86b734a380cd49ee19'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:57.467273 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-9d9dfc80-27a4-4b41-b70d-c3198e038acc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12a7c17b37ab40999cf11094f358f8ed', 'request_id': 'req-9d9dfc80-27a4-4b41-b70d-c3198e038acc'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:57.473167 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 925/1873] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:33:57 2022] DELETE /identity/v3/users/da9660bf705b4e86b734a380cd49ee19 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:57.582046 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-9d9dfc80-27a4-4b41-b70d-c3198e038acc admin admin] The token cache is being invalidate because project 12a7c17b37ab40999cf11094f358f8ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:33:57.582423 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-9d9dfc80-27a4-4b41-b70d-c3198e038acc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:57.583147 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-9d9dfc80-27a4-4b41-b70d-c3198e038acc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:57.587932 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 949/1874] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:33:57 2022] DELETE /identity/v3/projects/12a7c17b37ab40999cf11094f358f8ed => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:33:57.590015 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2662c-d68e-4de8-a32b-7e55d3fc0c82 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:57.592115 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-4bf2662c-d68e-4de8-a32b-7e55d3fc0c82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:57.668416 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2662c-d68e-4de8-a32b-7e55d3fc0c82 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:57.673169 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe377d-7d26-4576-8eeb-7279ab1431d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:57.674048 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe377d-7d26-4576-8eeb-7279ab1431d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:57.674335 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe377d-7d26-4576-8eeb-7279ab1431d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:57.674755 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe377d-7d26-4576-8eeb-7279ab1431d8 admin admin] PATH_INFO: `/v3/users/7cd9c89376814399ab81b7ad849f8484` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:57.676010 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffe377d-7d26-4576-8eeb-7279ab1431d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cd9c89376814399ab81b7ad849f8484)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:57.677942 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffe377d-7d26-4576-8eeb-7279ab1431d8 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:57.851770 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-5ffe377d-7d26-4576-8eeb-7279ab1431d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cd9c89376814399ab81b7ad849f8484'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:57.859407 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-5ffe377d-7d26-4576-8eeb-7279ab1431d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cd9c89376814399ab81b7ad849f8484'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:57.871748 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-5ffe377d-7d26-4576-8eeb-7279ab1431d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cd9c89376814399ab81b7ad849f8484'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:57.872448 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-5ffe377d-7d26-4576-8eeb-7279ab1431d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cd9c89376814399ab81b7ad849f8484'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:57.952816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9dfc80-27a4-4b41-b70d-c3198e038acc admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:57.954837 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-9d9dfc80-27a4-4b41-b70d-c3198e038acc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:57.970933 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-5ffe377d-7d26-4576-8eeb-7279ab1431d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cd9c89376814399ab81b7ad849f8484'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:57.981042 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 926/1875] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:33:57 2022] DELETE /identity/v3/users/7cd9c89376814399ab81b7ad849f8484 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:58.080701 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9dfc80-27a4-4b41-b70d-c3198e038acc admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:58.142665 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e17df-57d0-4b70-80dd-4fb359278508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:58.143718 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e17df-57d0-4b70-80dd-4fb359278508 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:58.144005 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e17df-57d0-4b70-80dd-4fb359278508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:58.144283 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e17df-57d0-4b70-80dd-4fb359278508 admin admin] PATH_INFO: `/v3/projects/e989357ca6a74fe1a93f986d3adb98ea` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:58.145482 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e17df-57d0-4b70-80dd-4fb359278508 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e989357ca6a74fe1a93f986d3adb98ea)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:58.147409 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e17df-57d0-4b70-80dd-4fb359278508 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:58.150016 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:58.280146 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-ed1e17df-57d0-4b70-80dd-4fb359278508 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e989357ca6a74fe1a93f986d3adb98ea', 'request_id': 'req-ed1e17df-57d0-4b70-80dd-4fb359278508'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:58.286659 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-ed1e17df-57d0-4b70-80dd-4fb359278508 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e989357ca6a74fe1a93f986d3adb98ea', 'request_id': 'req-ed1e17df-57d0-4b70-80dd-4fb359278508'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:58.294276 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-ed1e17df-57d0-4b70-80dd-4fb359278508 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e989357ca6a74fe1a93f986d3adb98ea', 'request_id': 'req-ed1e17df-57d0-4b70-80dd-4fb359278508'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:58.409991 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-ed1e17df-57d0-4b70-80dd-4fb359278508 admin admin] The token cache is being invalidate because project e989357ca6a74fe1a93f986d3adb98ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:33:58.410417 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-ed1e17df-57d0-4b70-80dd-4fb359278508 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:58.411086 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-ed1e17df-57d0-4b70-80dd-4fb359278508 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:58.412929 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 950/1876] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:33:57 2022] DELETE /identity/v3/projects/e989357ca6a74fe1a93f986d3adb98ea => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:33:58.418593 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ddda5f-8d85-4342-b825-640d7ba81d69 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:58.418897 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ddda5f-8d85-4342-b825-640d7ba81d69 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:58.419172 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ddda5f-8d85-4342-b825-640d7ba81d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:58.443420 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-c1ddda5f-8d85-4342-b825-640d7ba81d69 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:58.443420 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-c1ddda5f-8d85-4342-b825-640d7ba81d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:58.444601 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e17df-57d0-4b70-80dd-4fb359278508 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:58.445966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-ed1e17df-57d0-4b70-80dd-4fb359278508 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:58.460771 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:58.518883 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e17df-57d0-4b70-80dd-4fb359278508 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:58.523426 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 927/1877] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:58.531948 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367907bc-a91c-42c2-8bd0-e006397ad5bd None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:58.532282 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367907bc-a91c-42c2-8bd0-e006397ad5bd None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:58.532565 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367907bc-a91c-42c2-8bd0-e006397ad5bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:58.534494 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d189a94-a273-464d-87ba-e94eaa136a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:58.535603 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d189a94-a273-464d-87ba-e94eaa136a33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:58.535893 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d189a94-a273-464d-87ba-e94eaa136a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:58.536243 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d189a94-a273-464d-87ba-e94eaa136a33 admin admin] PATH_INFO: `/v3/projects/cc8188fbcf844769b24baec11728941c` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:58.537670 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d189a94-a273-464d-87ba-e94eaa136a33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc8188fbcf844769b24baec11728941c)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:58.540052 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d189a94-a273-464d-87ba-e94eaa136a33 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:58.560864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-367907bc-a91c-42c2-8bd0-e006397ad5bd None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:58.562067 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-367907bc-a91c-42c2-8bd0-e006397ad5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:58.570083 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 928/1878] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:33:58.574180 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367907bc-a91c-42c2-8bd0-e006397ad5bd None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:58.583975 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367907bc-a91c-42c2-8bd0-e006397ad5bd None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:58.589176 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f350f1-2c6c-4677-9423-9b116daa3df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:58.589808 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f350f1-2c6c-4677-9423-9b116daa3df1 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:58.590088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f350f1-2c6c-4677-9423-9b116daa3df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:58.590360 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f350f1-2c6c-4677-9423-9b116daa3df1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:58.591102 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f350f1-2c6c-4677-9423-9b116daa3df1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:33:58.591407 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f350f1-2c6c-4677-9423-9b116daa3df1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:58.593846 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f350f1-2c6c-4677-9423-9b116daa3df1 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:58.605155 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 929/1879] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:33:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:33:58.609623 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f350f1-2c6c-4677-9423-9b116daa3df1 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:58.620248 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f350f1-2c6c-4677-9423-9b116daa3df1 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:58.624923 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406aad0-fc75-4cb9-8848-26f40d843dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:58.625621 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406aad0-fc75-4cb9-8848-26f40d843dc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:58.625906 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406aad0-fc75-4cb9-8848-26f40d843dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:58.626186 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406aad0-fc75-4cb9-8848-26f40d843dc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:58.626866 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5406aad0-fc75-4cb9-8848-26f40d843dc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:58.628809 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5406aad0-fc75-4cb9-8848-26f40d843dc6 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:58.655829 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-2d189a94-a273-464d-87ba-e94eaa136a33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc8188fbcf844769b24baec11728941c', 'request_id': 'req-2d189a94-a273-464d-87ba-e94eaa136a33'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:58.661172 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-2d189a94-a273-464d-87ba-e94eaa136a33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc8188fbcf844769b24baec11728941c', 'request_id': 'req-2d189a94-a273-464d-87ba-e94eaa136a33'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:58.667278 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-2d189a94-a273-464d-87ba-e94eaa136a33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc8188fbcf844769b24baec11728941c', 'request_id': 'req-2d189a94-a273-464d-87ba-e94eaa136a33'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:58.786095 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-2d189a94-a273-464d-87ba-e94eaa136a33 admin admin] The token cache is being invalidate because project cc8188fbcf844769b24baec11728941c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:33:58.786297 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-2d189a94-a273-464d-87ba-e94eaa136a33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:58.786954 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-2d189a94-a273-464d-87ba-e94eaa136a33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:33:58.788692 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 930/1881] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:33:58 2022] POST /identity/v3/projects => generated 379 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:33:58.791948 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 951/1881] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:33:58 2022] DELETE /identity/v3/projects/cc8188fbcf844769b24baec11728941c => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:58.792910 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406aad0-fc75-4cb9-8848-26f40d843dc6 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:58.794517 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-5406aad0-fc75-4cb9-8848-26f40d843dc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:58.796332 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6f03c-aae0-486d-9b18-5daf278dfba0 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:58.797208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6f03c-aae0-486d-9b18-5daf278dfba0 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:58.797718 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6f03c-aae0-486d-9b18-5daf278dfba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:58.832873 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-aef6f03c-aae0-486d-9b18-5daf278dfba0 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:58.834085 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-aef6f03c-aae0-486d-9b18-5daf278dfba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:58.895827 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:58.938675 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406aad0-fc75-4cb9-8848-26f40d843dc6 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:58.944256 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a0774e-6237-4863-84bf-542b78a9cc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:58.945046 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0774e-6237-4863-84bf-542b78a9cc90 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:58.945454 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0774e-6237-4863-84bf-542b78a9cc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:58.945697 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0774e-6237-4863-84bf-542b78a9cc90 admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:58.946413 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a0774e-6237-4863-84bf-542b78a9cc90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:58.947828 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 952/1882] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:58.955596 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a0774e-6237-4863-84bf-542b78a9cc90 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:58.966842 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effed702-1cf1-4988-93c6-4a4e02050636 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:58.967282 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effed702-1cf1-4988-93c6-4a4e02050636 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:58.967633 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effed702-1cf1-4988-93c6-4a4e02050636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:59.044811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 931/1883] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:33:58 2022] POST /identity/v3/users => generated 290 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:33:59.045671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a0774e-6237-4863-84bf-542b78a9cc90 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:59.058544 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a0774e-6237-4863-84bf-542b78a9cc90 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:59.064509 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4ccf6-fb89-4a51-88b5-bc352ac0d96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:59.065202 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4ccf6-fb89-4a51-88b5-bc352ac0d96f admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:59.065706 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4ccf6-fb89-4a51-88b5-bc352ac0d96f admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:59.065992 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4ccf6-fb89-4a51-88b5-bc352ac0d96f admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:59.066826 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd4ccf6-fb89-4a51-88b5-bc352ac0d96f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:59.068752 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd4ccf6-fb89-4a51-88b5-bc352ac0d96f admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:59.087533 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 932/1884] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:33:59.095163 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4ccf6-fb89-4a51-88b5-bc352ac0d96f admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:59.121632 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4ccf6-fb89-4a51-88b5-bc352ac0d96f admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:59.132043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-effed702-1cf1-4988-93c6-4a4e02050636 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:59.133563 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-effed702-1cf1-4988-93c6-4a4e02050636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:59.143428 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 953/1885] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:59.146071 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacfae76-c141-4773-bb8d-2cc217b514e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:59.147105 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacfae76-c141-4773-bb8d-2cc217b514e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:59.147396 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacfae76-c141-4773-bb8d-2cc217b514e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:59.147592 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effed702-1cf1-4988-93c6-4a4e02050636 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:59.149839 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacfae76-c141-4773-bb8d-2cc217b514e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:59.150605 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacfae76-c141-4773-bb8d-2cc217b514e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:59.152476 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacfae76-c141-4773-bb8d-2cc217b514e4 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:59.158162 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effed702-1cf1-4988-93c6-4a4e02050636 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:59.163521 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa300d-a453-4292-8644-d0b2b97e20a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:59.164443 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa300d-a453-4292-8644-d0b2b97e20a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:59.164805 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa300d-a453-4292-8644-d0b2b97e20a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:59.167907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa300d-a453-4292-8644-d0b2b97e20a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:59.168039 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 933/1886] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:59 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:33:59.168135 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fa300d-a453-4292-8644-d0b2b97e20a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:33:59.168135 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fa300d-a453-4292-8644-d0b2b97e20a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:59.168474 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fa300d-a453-4292-8644-d0b2b97e20a6 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:59.175711 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacfae76-c141-4773-bb8d-2cc217b514e4 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:59.183005 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 954/1887] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:33:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:33:59.187932 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa300d-a453-4292-8644-d0b2b97e20a6 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:59.193624 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacfae76-c141-4773-bb8d-2cc217b514e4 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:59.196364 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97824e-c19f-4d01-b8a5-e2803a8da741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:59.197516 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97824e-c19f-4d01-b8a5-e2803a8da741 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:59.197981 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97824e-c19f-4d01-b8a5-e2803a8da741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:59.198457 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97824e-c19f-4d01-b8a5-e2803a8da741 admin admin] PATH_INFO: `/v3/projects/68c7fa0bbe3342f6b6a2d48cab172f2e/users/98ebd375e9c548de871c6251b0f9c27c/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:59.200142 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa300d-a453-4292-8644-d0b2b97e20a6 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:59.209200 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef370d-8616-433c-bde0-559d980a76e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:59.210670 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef370d-8616-433c-bde0-559d980a76e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:59.210955 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef370d-8616-433c-bde0-559d980a76e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:59.211230 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef370d-8616-433c-bde0-559d980a76e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:59.211998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ef370d-8616-433c-bde0-559d980a76e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:59.216581 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ef370d-8616-433c-bde0-559d980a76e1 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:59.230365 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e97824e-c19f-4d01-b8a5-e2803a8da741 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68c7fa0bbe3342f6b6a2d48cab172f2e, user_id=98ebd375e9c548de871c6251b0f9c27c, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:59.234458 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e97824e-c19f-4d01-b8a5-e2803a8da741 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:59.373975 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 955/1888] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:33:59 2022] POST /identity/v3/projects => generated 385 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:33:59.374956 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef370d-8616-433c-bde0-559d980a76e1 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:59.377238 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 934/1889] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:33:59 2022] PUT /identity/v3/projects/68c7fa0bbe3342f6b6a2d48cab172f2e/users/98ebd375e9c548de871c6251b0f9c27c/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:33:59.390301 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd7310-a994-4c61-9599-2ef69de86fe2 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:59.390586 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd7310-a994-4c61-9599-2ef69de86fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:59.390859 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd7310-a994-4c61-9599-2ef69de86fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:59.422272 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-f2bd7310-a994-4c61-9599-2ef69de86fe2 None None] MFA Rules not processed for user `98ebd375e9c548de871c6251b0f9c27c`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:59.424454 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-f2bd7310-a994-4c61-9599-2ef69de86fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:33:59.455423 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef370d-8616-433c-bde0-559d980a76e1 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:59.461196 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:33:59.465363 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accf5b21-b81d-46d8-b8d2-283d706b12da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:59.470409 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf5b21-b81d-46d8-b8d2-283d706b12da admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:59.470409 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf5b21-b81d-46d8-b8d2-283d706b12da admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:59.470409 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf5b21-b81d-46d8-b8d2-283d706b12da admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:59.470409 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accf5b21-b81d-46d8-b8d2-283d706b12da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:59.470409 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accf5b21-b81d-46d8-b8d2-283d706b12da admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:59.510820 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 935/1890] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:33:59.515743 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd7310-a994-4c61-9599-2ef69de86fe2 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:59.528594 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd7310-a994-4c61-9599-2ef69de86fe2 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:59.534237 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f17dcb8-fb37-4a22-868a-54f783b7f77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:59.534895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17dcb8-fb37-4a22-868a-54f783b7f77a admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:59.535178 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17dcb8-fb37-4a22-868a-54f783b7f77a admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:59.535456 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17dcb8-fb37-4a22-868a-54f783b7f77a admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:59.536183 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f17dcb8-fb37-4a22-868a-54f783b7f77a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:59.538077 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f17dcb8-fb37-4a22-868a-54f783b7f77a admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:59.555996 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 956/1891] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:33:59 2022] POST /identity/v3/users => generated 293 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:33:59.561831 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accf5b21-b81d-46d8-b8d2-283d706b12da admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:59.572471 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accf5b21-b81d-46d8-b8d2-283d706b12da admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:59.577918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7523f6-bdbd-4d9f-aa66-a37309a7399c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:59.578583 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7523f6-bdbd-4d9f-aa66-a37309a7399c admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:59.578868 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7523f6-bdbd-4d9f-aa66-a37309a7399c admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:59.579171 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7523f6-bdbd-4d9f-aa66-a37309a7399c admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:59.582191 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7523f6-bdbd-4d9f-aa66-a37309a7399c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:59.585535 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7523f6-bdbd-4d9f-aa66-a37309a7399c admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:59.599499 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 957/1892] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:33:59.603257 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7523f6-bdbd-4d9f-aa66-a37309a7399c admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:59.616466 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7523f6-bdbd-4d9f-aa66-a37309a7399c admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:59.622379 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b62afd8-20f6-4917-b9e8-10f84de6fcde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:59.623591 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62afd8-20f6-4917-b9e8-10f84de6fcde admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:59.623882 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62afd8-20f6-4917-b9e8-10f84de6fcde admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:59.624156 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62afd8-20f6-4917-b9e8-10f84de6fcde admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:59.625019 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b62afd8-20f6-4917-b9e8-10f84de6fcde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:59.626586 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b62afd8-20f6-4917-b9e8-10f84de6fcde admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:59.637014 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 958/1893] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:33:59.641612 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b62afd8-20f6-4917-b9e8-10f84de6fcde admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:59.652202 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 936/1894] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:33:59 2022] POST /identity/v3/projects => generated 381 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:33:59.653258 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b62afd8-20f6-4917-b9e8-10f84de6fcde admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:59.656722 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f17dcb8-fb37-4a22-868a-54f783b7f77a admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:59.749979 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f17dcb8-fb37-4a22-868a-54f783b7f77a admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:59.750234 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f96ae2a-d986-4c90-882f-40fdba5ef27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:59.751113 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f96ae2a-d986-4c90-882f-40fdba5ef27f admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:59.751864 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f96ae2a-d986-4c90-882f-40fdba5ef27f admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:59.752341 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f96ae2a-d986-4c90-882f-40fdba5ef27f admin admin] PATH_INFO: `/v3/projects/8384c1faa0fa4d1c89b25ffa5f252974/users/e24570ae7f4d41649b764a0a9318ccc1/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:59.755097 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbbf2dc-15dd-457e-a694-8d84a58086eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:59.755814 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbf2dc-15dd-457e-a694-8d84a58086eb admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:59.756084 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbf2dc-15dd-457e-a694-8d84a58086eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:59.756344 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbf2dc-15dd-457e-a694-8d84a58086eb admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:59.757592 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbbf2dc-15dd-457e-a694-8d84a58086eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:59.759040 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbbf2dc-15dd-457e-a694-8d84a58086eb admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:59.770521 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f96ae2a-d986-4c90-882f-40fdba5ef27f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8384c1faa0fa4d1c89b25ffa5f252974, user_id=e24570ae7f4d41649b764a0a9318ccc1, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:59.773239 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f96ae2a-d986-4c90-882f-40fdba5ef27f admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:59.854010 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 937/1895] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:33:59 2022] POST /identity/v3/users => generated 297 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:33:59.858379 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbbf2dc-15dd-457e-a694-8d84a58086eb admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:59.868966 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbbf2dc-15dd-457e-a694-8d84a58086eb admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:59.874671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637790d-ec7a-4fe7-ae52-bd3ae38faf78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:33:59.875355 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637790d-ec7a-4fe7-ae52-bd3ae38faf78 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:59.875841 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637790d-ec7a-4fe7-ae52-bd3ae38faf78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:59.875996 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637790d-ec7a-4fe7-ae52-bd3ae38faf78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:59.876681 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c637790d-ec7a-4fe7-ae52-bd3ae38faf78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:33:59.878333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c637790d-ec7a-4fe7-ae52-bd3ae38faf78 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:33:59.883647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 959/1896] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:33:59 2022] PUT /identity/v3/projects/8384c1faa0fa4d1c89b25ffa5f252974/users/e24570ae7f4d41649b764a0a9318ccc1/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:33:59.895948 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b638124-534d-4a3d-88a0-6a5b60abd47a None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:33:59.896258 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b638124-534d-4a3d-88a0-6a5b60abd47a None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:33:59.896536 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b638124-534d-4a3d-88a0-6a5b60abd47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:33:59.904043 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 938/1897] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:33:59 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:33:59.907007 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637790d-ec7a-4fe7-ae52-bd3ae38faf78 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:33:59.917821 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637790d-ec7a-4fe7-ae52-bd3ae38faf78 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:33:59.929908 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-5b638124-534d-4a3d-88a0-6a5b60abd47a None None] MFA Rules not processed for user `e24570ae7f4d41649b764a0a9318ccc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:33:59.931221 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-5b638124-534d-4a3d-88a0-6a5b60abd47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:00.004351 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dcd230-da9e-4168-904c-7d1ea7171f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:00.005320 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dcd230-da9e-4168-904c-7d1ea7171f85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:00.005664 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dcd230-da9e-4168-904c-7d1ea7171f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:00.006005 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dcd230-da9e-4168-904c-7d1ea7171f85 admin admin] PATH_INFO: `/v3/projects/4b7b566a34fb4a8e8c186e87d351130c/users/723ceddaca17406d96914c0a73b3da72/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:00.015330 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 960/1898] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:33:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:34:00.021355 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dcd230-da9e-4168-904c-7d1ea7171f85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b7b566a34fb4a8e8c186e87d351130c, user_id=723ceddaca17406d96914c0a73b3da72, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:00.024501 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dcd230-da9e-4168-904c-7d1ea7171f85 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:00.024911 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b638124-534d-4a3d-88a0-6a5b60abd47a None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:00.035792 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b638124-534d-4a3d-88a0-6a5b60abd47a None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:00.040725 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c40bc82-8928-4c82-83c3-86ae6333d9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:00.041383 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c40bc82-8928-4c82-83c3-86ae6333d9ff admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:00.041702 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c40bc82-8928-4c82-83c3-86ae6333d9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:00.041985 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c40bc82-8928-4c82-83c3-86ae6333d9ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:00.042737 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c40bc82-8928-4c82-83c3-86ae6333d9ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:00.045169 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c40bc82-8928-4c82-83c3-86ae6333d9ff admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:00.124577 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:00.134766 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 939/1899] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:33:59 2022] PUT /identity/v3/projects/4b7b566a34fb4a8e8c186e87d351130c/users/723ceddaca17406d96914c0a73b3da72/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:34:00.138921 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dcd230-da9e-4168-904c-7d1ea7171f85 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:00.157836 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 961/1900] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:34:00 2022] POST /identity/v3/projects => generated 385 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:34:00.159298 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c40bc82-8928-4c82-83c3-86ae6333d9ff admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:00.258465 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dcd230-da9e-4168-904c-7d1ea7171f85 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:00.272627 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c40bc82-8928-4c82-83c3-86ae6333d9ff admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:00.284473 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29423b20-dc69-43e5-b790-31da0f991f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:00.285946 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29423b20-dc69-43e5-b790-31da0f991f38 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:00.286395 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29423b20-dc69-43e5-b790-31da0f991f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:00.287218 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29423b20-dc69-43e5-b790-31da0f991f38 admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:00.289159 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29423b20-dc69-43e5-b790-31da0f991f38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:00.291382 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29423b20-dc69-43e5-b790-31da0f991f38 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:00.350412 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9424f50-7ef3-4c50-b307-0013fb894b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:00.351201 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9424f50-7ef3-4c50-b307-0013fb894b1f admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:00.351595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9424f50-7ef3-4c50-b307-0013fb894b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:00.352076 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9424f50-7ef3-4c50-b307-0013fb894b1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:00.352945 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9424f50-7ef3-4c50-b307-0013fb894b1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:00.355025 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9424f50-7ef3-4c50-b307-0013fb894b1f admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:00.375236 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 940/1901] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:34:00.376370 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9424f50-7ef3-4c50-b307-0013fb894b1f admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:00.390051 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:00.390924 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 962/1902] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:34:00 2022] POST /identity/v3/users => generated 299 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:34:00.395719 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9424f50-7ef3-4c50-b307-0013fb894b1f admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:00.397630 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29423b20-dc69-43e5-b790-31da0f991f38 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:00.412916 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29423b20-dc69-43e5-b790-31da0f991f38 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:00.419436 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8844eade-860a-41b9-9648-4f1865c68b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:00.421049 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844eade-860a-41b9-9648-4f1865c68b9f admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:00.423408 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844eade-860a-41b9-9648-4f1865c68b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:00.423408 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844eade-860a-41b9-9648-4f1865c68b9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:00.424111 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8844eade-860a-41b9-9648-4f1865c68b9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:00.424227 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41aa7fb-fd21-44b3-b3ff-f4f8cf5bdaf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:00.425148 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41aa7fb-fd21-44b3-b3ff-f4f8cf5bdaf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:00.426173 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8844eade-860a-41b9-9648-4f1865c68b9f admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:00.428318 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41aa7fb-fd21-44b3-b3ff-f4f8cf5bdaf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:00.428628 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41aa7fb-fd21-44b3-b3ff-f4f8cf5bdaf6 admin admin] PATH_INFO: `/v3/projects/4b7b566a34fb4a8e8c186e87d351130c/users/723ceddaca17406d96914c0a73b3da72/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:00.431741 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41aa7fb-fd21-44b3-b3ff-f4f8cf5bdaf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b7b566a34fb4a8e8c186e87d351130c, user_id=723ceddaca17406d96914c0a73b3da72, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:00.435346 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41aa7fb-fd21-44b3-b3ff-f4f8cf5bdaf6 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:00.460317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 941/1903] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:34:00 2022] PUT /identity/v3/projects/4b7b566a34fb4a8e8c186e87d351130c/users/723ceddaca17406d96914c0a73b3da72/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:00.460577 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 963/1904] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:34:00.471804 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41aa7fb-fd21-44b3-b3ff-f4f8cf5bdaf6 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:00.479611 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c460a-9e0f-40fa-a3ac-46b9340bfb4b None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:00.479611 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c460a-9e0f-40fa-a3ac-46b9340bfb4b None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:00.480967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c460a-9e0f-40fa-a3ac-46b9340bfb4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:00.529976 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-b81c460a-9e0f-40fa-a3ac-46b9340bfb4b None None] MFA Rules not processed for user `723ceddaca17406d96914c0a73b3da72`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:00.531409 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-b81c460a-9e0f-40fa-a3ac-46b9340bfb4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:00.575546 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41aa7fb-fd21-44b3-b3ff-f4f8cf5bdaf6 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:00.584914 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b8d9c-7bd8-4ffd-bf28-25ad763e9894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:00.587998 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b8d9c-7bd8-4ffd-bf28-25ad763e9894 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:00.588307 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b8d9c-7bd8-4ffd-bf28-25ad763e9894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:00.590275 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b8d9c-7bd8-4ffd-bf28-25ad763e9894 admin admin] PATH_INFO: `/v3/projects/8e9256a368054ed5b20d1cf1ca9bfd15/users/97c8f68ae3aa4fa280228c2b86390fbd/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:00.631869 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b8d9c-7bd8-4ffd-bf28-25ad763e9894 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e9256a368054ed5b20d1cf1ca9bfd15, user_id=97c8f68ae3aa4fa280228c2b86390fbd, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:00.631869 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b8d9c-7bd8-4ffd-bf28-25ad763e9894 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:00.684441 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 964/1905] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:00 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:00.732430 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81c460a-9e0f-40fa-a3ac-46b9340bfb4b None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:00.733947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-b81c460a-9e0f-40fa-a3ac-46b9340bfb4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:00.751542 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 942/1906] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:34:00 2022] PUT /identity/v3/projects/8e9256a368054ed5b20d1cf1ca9bfd15/users/97c8f68ae3aa4fa280228c2b86390fbd/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:00.754919 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b8d9c-7bd8-4ffd-bf28-25ad763e9894 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:00.827322 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81c460a-9e0f-40fa-a3ac-46b9340bfb4b None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:00.887004 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf2fed-5253-42ac-a331-325915ff56e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:00.887763 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf2fed-5253-42ac-a331-325915ff56e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:00.888159 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf2fed-5253-42ac-a331-325915ff56e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:00.889093 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf2fed-5253-42ac-a331-325915ff56e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:00.927343 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b8d9c-7bd8-4ffd-bf28-25ad763e9894 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:00.932485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a97fc4e-dce0-451c-b4c1-c409b26eb1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:00.933907 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97fc4e-dce0-451c-b4c1-c409b26eb1f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:00.933907 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97fc4e-dce0-451c-b4c1-c409b26eb1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:00.934107 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97fc4e-dce0-451c-b4c1-c409b26eb1f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:00.935117 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a97fc4e-dce0-451c-b4c1-c409b26eb1f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:00.937294 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a97fc4e-dce0-451c-b4c1-c409b26eb1f1 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:00.959526 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 943/1907] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:34:00.960887 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a97fc4e-dce0-451c-b4c1-c409b26eb1f1 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:00.967306 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf2fed-5253-42ac-a331-325915ff56e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:00.968690 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf2fed-5253-42ac-a331-325915ff56e3 service cinder] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:00.979488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 965/1908] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:34:00 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:34:00.985190 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a97fc4e-dce0-451c-b4c1-c409b26eb1f1 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:01.001164 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a5817f-778b-4290-8ef6-c04610a685fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:01.004722 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5817f-778b-4290-8ef6-c04610a685fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:01.005280 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5817f-778b-4290-8ef6-c04610a685fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:01.005280 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5817f-778b-4290-8ef6-c04610a685fc admin admin] PATH_INFO: `/v3/projects/8e9256a368054ed5b20d1cf1ca9bfd15/users/97c8f68ae3aa4fa280228c2b86390fbd/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:01.007809 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a5817f-778b-4290-8ef6-c04610a685fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e9256a368054ed5b20d1cf1ca9bfd15, user_id=97c8f68ae3aa4fa280228c2b86390fbd, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:01.011592 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a5817f-778b-4290-8ef6-c04610a685fc admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:01.023188 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 944/1909] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:34:00 2022] PUT /identity/v3/projects/8e9256a368054ed5b20d1cf1ca9bfd15/users/97c8f68ae3aa4fa280228c2b86390fbd/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:34:01.034554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb67bbdb-301f-4e22-8697-48b7100ef051 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:01.035055 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb67bbdb-301f-4e22-8697-48b7100ef051 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:01.035164 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb67bbdb-301f-4e22-8697-48b7100ef051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:01.070417 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-cb67bbdb-301f-4e22-8697-48b7100ef051 None None] MFA Rules not processed for user `97c8f68ae3aa4fa280228c2b86390fbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:01.079527 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-cb67bbdb-301f-4e22-8697-48b7100ef051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:01.178813 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 966/1910] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:01.184751 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:01.247628 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:01.331651 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a5817f-778b-4290-8ef6-c04610a685fc admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:01.331651 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-12a5817f-778b-4290-8ef6-c04610a685fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:01.402254 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a5817f-778b-4290-8ef6-c04610a685fc admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:01.405921 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a405affe-36ed-4b88-84e8-94307bbda0c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:01.406664 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405affe-36ed-4b88-84e8-94307bbda0c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:01.406937 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405affe-36ed-4b88-84e8-94307bbda0c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:01.407212 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405affe-36ed-4b88-84e8-94307bbda0c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:01.417335 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a405affe-36ed-4b88-84e8-94307bbda0c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:01.418778 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a405affe-36ed-4b88-84e8-94307bbda0c5 service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:01.430760 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 945/1911] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:34:01 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:34:01.461205 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb67bbdb-301f-4e22-8697-48b7100ef051 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:01.542360 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb67bbdb-301f-4e22-8697-48b7100ef051 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:01.553444 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35662a05-0958-4d65-8ef9-ce659c098800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:01.554956 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35662a05-0958-4d65-8ef9-ce659c098800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:01.555236 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35662a05-0958-4d65-8ef9-ce659c098800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:01.555511 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35662a05-0958-4d65-8ef9-ce659c098800 admin admin] PATH_INFO: `/v3/users/e24570ae7f4d41649b764a0a9318ccc1` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:01.556790 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35662a05-0958-4d65-8ef9-ce659c098800 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e24570ae7f4d41649b764a0a9318ccc1)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:01.558716 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35662a05-0958-4d65-8ef9-ce659c098800 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:01.587192 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:01.772094 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-35662a05-0958-4d65-8ef9-ce659c098800 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e24570ae7f4d41649b764a0a9318ccc1'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:01.773436 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-35662a05-0958-4d65-8ef9-ce659c098800 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e24570ae7f4d41649b764a0a9318ccc1'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:01.783628 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-35662a05-0958-4d65-8ef9-ce659c098800 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e24570ae7f4d41649b764a0a9318ccc1'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:01.784382 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-35662a05-0958-4d65-8ef9-ce659c098800 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e24570ae7f4d41649b764a0a9318ccc1'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:01.798764 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-35662a05-0958-4d65-8ef9-ce659c098800 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e24570ae7f4d41649b764a0a9318ccc1'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:01.881856 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 967/1912] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:34:01 2022] DELETE /identity/v3/users/e24570ae7f4d41649b764a0a9318ccc1 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:01.886148 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898712e-7a24-4c11-a2a6-8decda163509 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:01.886440 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898712e-7a24-4c11-a2a6-8decda163509 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:01.886773 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898712e-7a24-4c11-a2a6-8decda163509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:01.907666 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-c898712e-7a24-4c11-a2a6-8decda163509 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:01.909057 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-c898712e-7a24-4c11-a2a6-8decda163509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:01.982964 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 946/1913] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:34:01.995129 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35662a05-0958-4d65-8ef9-ce659c098800 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:01.997356 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-35662a05-0958-4d65-8ef9-ce659c098800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:02.014993 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:02.072819 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35662a05-0958-4d65-8ef9-ce659c098800 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:02.076494 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a554d4f0-fa9f-4d63-b902-cdd35fd6804c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:02.077239 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554d4f0-fa9f-4d63-b902-cdd35fd6804c service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:02.077537 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554d4f0-fa9f-4d63-b902-cdd35fd6804c service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:02.077817 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554d4f0-fa9f-4d63-b902-cdd35fd6804c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:02.087555 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a554d4f0-fa9f-4d63-b902-cdd35fd6804c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:02.089027 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a554d4f0-fa9f-4d63-b902-cdd35fd6804c service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:02.105825 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 968/1914] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:34:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:34:02.224474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c898712e-7a24-4c11-a2a6-8decda163509 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:02.227536 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-c898712e-7a24-4c11-a2a6-8decda163509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:02.251362 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c898712e-7a24-4c11-a2a6-8decda163509 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:02.258203 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d109c0-22b5-46f0-ae5f-3989f4937bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:02.259331 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d109c0-22b5-46f0-ae5f-3989f4937bfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:02.259496 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d109c0-22b5-46f0-ae5f-3989f4937bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:02.259846 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d109c0-22b5-46f0-ae5f-3989f4937bfd admin admin] PATH_INFO: `/v3/users/98ebd375e9c548de871c6251b0f9c27c` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:02.261168 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d109c0-22b5-46f0-ae5f-3989f4937bfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=98ebd375e9c548de871c6251b0f9c27c)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:02.263852 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d109c0-22b5-46f0-ae5f-3989f4937bfd admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:02.302900 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:02.639246 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-98d109c0-22b5-46f0-ae5f-3989f4937bfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98ebd375e9c548de871c6251b0f9c27c'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:02.652381 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-98d109c0-22b5-46f0-ae5f-3989f4937bfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98ebd375e9c548de871c6251b0f9c27c'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:02.674211 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-98d109c0-22b5-46f0-ae5f-3989f4937bfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98ebd375e9c548de871c6251b0f9c27c'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:02.674903 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-98d109c0-22b5-46f0-ae5f-3989f4937bfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98ebd375e9c548de871c6251b0f9c27c'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:02.739465 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-98d109c0-22b5-46f0-ae5f-3989f4937bfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98ebd375e9c548de871c6251b0f9c27c'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:02.751656 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 947/1915] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:34:02 2022] DELETE /identity/v3/users/98ebd375e9c548de871c6251b0f9c27c => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:02.756647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19249e58-32f8-4e84-93c9-52da27998b4c None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:02.757019 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19249e58-32f8-4e84-93c9-52da27998b4c None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:02.757399 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19249e58-32f8-4e84-93c9-52da27998b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:02.776593 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-19249e58-32f8-4e84-93c9-52da27998b4c None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:02.777959 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-19249e58-32f8-4e84-93c9-52da27998b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:02.825268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d109c0-22b5-46f0-ae5f-3989f4937bfd admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:02.827132 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-98d109c0-22b5-46f0-ae5f-3989f4937bfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:02.924741 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 969/1916] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:02.948111 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d109c0-22b5-46f0-ae5f-3989f4937bfd admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:02.953738 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670dce44-02aa-40d6-89b1-649c07156ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:02.954844 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670dce44-02aa-40d6-89b1-649c07156ca6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:02.955196 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670dce44-02aa-40d6-89b1-649c07156ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:02.955537 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670dce44-02aa-40d6-89b1-649c07156ca6 admin admin] PATH_INFO: `/v3/projects/8384c1faa0fa4d1c89b25ffa5f252974` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:02.961159 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670dce44-02aa-40d6-89b1-649c07156ca6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8384c1faa0fa4d1c89b25ffa5f252974)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:02.963825 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670dce44-02aa-40d6-89b1-649c07156ca6 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:02.967658 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19249e58-32f8-4e84-93c9-52da27998b4c None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:02.971500 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-19249e58-32f8-4e84-93c9-52da27998b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:03.073134 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19249e58-32f8-4e84-93c9-52da27998b4c None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:03.078715 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a214f98-bb86-4280-8516-edc6fdb6eb5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:03.079789 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a214f98-bb86-4280-8516-edc6fdb6eb5d service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:03.080265 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a214f98-bb86-4280-8516-edc6fdb6eb5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:03.080774 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a214f98-bb86-4280-8516-edc6fdb6eb5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:03.095073 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a214f98-bb86-4280-8516-edc6fdb6eb5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:03.097583 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a214f98-bb86-4280-8516-edc6fdb6eb5d service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:03.109710 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 970/1917] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:34:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:34:03.124231 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-670dce44-02aa-40d6-89b1-649c07156ca6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8384c1faa0fa4d1c89b25ffa5f252974', 'request_id': 'req-670dce44-02aa-40d6-89b1-649c07156ca6'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:03.133031 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-670dce44-02aa-40d6-89b1-649c07156ca6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8384c1faa0fa4d1c89b25ffa5f252974', 'request_id': 'req-670dce44-02aa-40d6-89b1-649c07156ca6'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:03.150265 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-670dce44-02aa-40d6-89b1-649c07156ca6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8384c1faa0fa4d1c89b25ffa5f252974', 'request_id': 'req-670dce44-02aa-40d6-89b1-649c07156ca6'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:03.298250 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-670dce44-02aa-40d6-89b1-649c07156ca6 admin admin] The token cache is being invalidate because project 8384c1faa0fa4d1c89b25ffa5f252974 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:34:03.299789 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-670dce44-02aa-40d6-89b1-649c07156ca6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:03.299789 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-670dce44-02aa-40d6-89b1-649c07156ca6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:03.301001 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 948/1918] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:34:02 2022] DELETE /identity/v3/projects/8384c1faa0fa4d1c89b25ffa5f252974 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:03.305283 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a214f98-bb86-4280-8516-edc6fdb6eb5d service neutron] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:03.315961 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a214f98-bb86-4280-8516-edc6fdb6eb5d service neutron] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:03.398807 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf753426-29a9-4be2-9d50-f89f3e631d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:03.403230 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf753426-29a9-4be2-9d50-f89f3e631d29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:03.403522 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf753426-29a9-4be2-9d50-f89f3e631d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:03.406444 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf753426-29a9-4be2-9d50-f89f3e631d29 admin admin] PATH_INFO: `/v3/users/97c8f68ae3aa4fa280228c2b86390fbd` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:03.408988 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf753426-29a9-4be2-9d50-f89f3e631d29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97c8f68ae3aa4fa280228c2b86390fbd)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:03.412057 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf753426-29a9-4be2-9d50-f89f3e631d29 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:03.414063 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:03.664483 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-cf753426-29a9-4be2-9d50-f89f3e631d29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97c8f68ae3aa4fa280228c2b86390fbd'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:03.667813 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-cf753426-29a9-4be2-9d50-f89f3e631d29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97c8f68ae3aa4fa280228c2b86390fbd'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:03.686543 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-cf753426-29a9-4be2-9d50-f89f3e631d29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97c8f68ae3aa4fa280228c2b86390fbd'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:03.688544 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-cf753426-29a9-4be2-9d50-f89f3e631d29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97c8f68ae3aa4fa280228c2b86390fbd'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:03.710822 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-cf753426-29a9-4be2-9d50-f89f3e631d29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97c8f68ae3aa4fa280228c2b86390fbd'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:03.747181 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670dce44-02aa-40d6-89b1-649c07156ca6 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:03.748841 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-670dce44-02aa-40d6-89b1-649c07156ca6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:03.763892 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670dce44-02aa-40d6-89b1-649c07156ca6 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:03.780875 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a0657-ffd3-4ff5-aa94-5b6645c729d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:03.780875 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a0657-ffd3-4ff5-aa94-5b6645c729d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:03.780875 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a0657-ffd3-4ff5-aa94-5b6645c729d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:03.780875 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a0657-ffd3-4ff5-aa94-5b6645c729d3 admin admin] PATH_INFO: `/v3/projects/68c7fa0bbe3342f6b6a2d48cab172f2e` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:03.781302 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214a0657-ffd3-4ff5-aa94-5b6645c729d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68c7fa0bbe3342f6b6a2d48cab172f2e)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:03.781302 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214a0657-ffd3-4ff5-aa94-5b6645c729d3 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:03.804854 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:03.827587 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 971/1919] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:34:03 2022] DELETE /identity/v3/users/97c8f68ae3aa4fa280228c2b86390fbd => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:03.918293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-214a0657-ffd3-4ff5-aa94-5b6645c729d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68c7fa0bbe3342f6b6a2d48cab172f2e', 'request_id': 'req-214a0657-ffd3-4ff5-aa94-5b6645c729d3'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:03.928780 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-214a0657-ffd3-4ff5-aa94-5b6645c729d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68c7fa0bbe3342f6b6a2d48cab172f2e', 'request_id': 'req-214a0657-ffd3-4ff5-aa94-5b6645c729d3'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:03.943978 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-214a0657-ffd3-4ff5-aa94-5b6645c729d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68c7fa0bbe3342f6b6a2d48cab172f2e', 'request_id': 'req-214a0657-ffd3-4ff5-aa94-5b6645c729d3'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:03.991871 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf753426-29a9-4be2-9d50-f89f3e631d29 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:03.993540 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-cf753426-29a9-4be2-9d50-f89f3e631d29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:04.032617 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-214a0657-ffd3-4ff5-aa94-5b6645c729d3 admin admin] The token cache is being invalidate because project 68c7fa0bbe3342f6b6a2d48cab172f2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:34:04.032912 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-214a0657-ffd3-4ff5-aa94-5b6645c729d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:04.034075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-214a0657-ffd3-4ff5-aa94-5b6645c729d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:04.036836 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 949/1920] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:34:03 2022] DELETE /identity/v3/projects/68c7fa0bbe3342f6b6a2d48cab172f2e => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:34:04.041159 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a0657-ffd3-4ff5-aa94-5b6645c729d3 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:04.047606 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-214a0657-ffd3-4ff5-aa94-5b6645c729d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:04.150386 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf753426-29a9-4be2-9d50-f89f3e631d29 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:04.167955 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a0657-ffd3-4ff5-aa94-5b6645c729d3 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:04.174901 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d9c8f-1a84-46b9-9bb7-3bcf795e0154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:04.175949 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d9c8f-1a84-46b9-9bb7-3bcf795e0154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:04.177293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d9c8f-1a84-46b9-9bb7-3bcf795e0154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:04.177293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d9c8f-1a84-46b9-9bb7-3bcf795e0154 admin admin] PATH_INFO: `/v3/users/723ceddaca17406d96914c0a73b3da72` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:04.178897 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20d9c8f-1a84-46b9-9bb7-3bcf795e0154 admin admin] RBAC: Authorizing `identity:delete_user(user_id=723ceddaca17406d96914c0a73b3da72)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:04.181304 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20d9c8f-1a84-46b9-9bb7-3bcf795e0154 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:04.261980 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d2440-300b-4419-84aa-49e9a3b9529e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:04.262898 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d2440-300b-4419-84aa-49e9a3b9529e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:04.263365 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d2440-300b-4419-84aa-49e9a3b9529e admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:04.263733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d2440-300b-4419-84aa-49e9a3b9529e admin admin] PATH_INFO: `/v3/users/6a0724b351f04577a22a143fe96e26c4` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:04.266616 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992d2440-300b-4419-84aa-49e9a3b9529e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a0724b351f04577a22a143fe96e26c4)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:04.270023 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992d2440-300b-4419-84aa-49e9a3b9529e admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:04.399652 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:04.450351 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-a20d9c8f-1a84-46b9-9bb7-3bcf795e0154 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '723ceddaca17406d96914c0a73b3da72'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:04.477363 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-a20d9c8f-1a84-46b9-9bb7-3bcf795e0154 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '723ceddaca17406d96914c0a73b3da72'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:04.517587 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-a20d9c8f-1a84-46b9-9bb7-3bcf795e0154 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '723ceddaca17406d96914c0a73b3da72'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:04.518932 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-a20d9c8f-1a84-46b9-9bb7-3bcf795e0154 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '723ceddaca17406d96914c0a73b3da72'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:04.554120 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-992d2440-300b-4419-84aa-49e9a3b9529e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a0724b351f04577a22a143fe96e26c4'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:04.554868 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-992d2440-300b-4419-84aa-49e9a3b9529e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a0724b351f04577a22a143fe96e26c4'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:04.581031 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-992d2440-300b-4419-84aa-49e9a3b9529e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a0724b351f04577a22a143fe96e26c4'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:04.581789 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-992d2440-300b-4419-84aa-49e9a3b9529e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a0724b351f04577a22a143fe96e26c4'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:04.617802 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-992d2440-300b-4419-84aa-49e9a3b9529e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a0724b351f04577a22a143fe96e26c4'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:04.635227 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-a20d9c8f-1a84-46b9-9bb7-3bcf795e0154 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '723ceddaca17406d96914c0a73b3da72'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:04.648903 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 950/1921] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:34:04 2022] DELETE /identity/v3/users/723ceddaca17406d96914c0a73b3da72 => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:04.650095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d9c8f-1a84-46b9-9bb7-3bcf795e0154 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:04.655795 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-a20d9c8f-1a84-46b9-9bb7-3bcf795e0154 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:04.723182 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 972/1922] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:34:03 2022] DELETE /identity/v3/users/6a0724b351f04577a22a143fe96e26c4 => generated 0 bytes in 732 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:04.727872 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3fb2d-acf6-4b50-979a-f8c353a39378 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:04.728162 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3fb2d-acf6-4b50-979a-f8c353a39378 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:04.728439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3fb2d-acf6-4b50-979a-f8c353a39378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:04.736261 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:04.772421 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-35f3fb2d-acf6-4b50-979a-f8c353a39378 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:04.773797 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-35f3fb2d-acf6-4b50-979a-f8c353a39378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:04.786984 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d9c8f-1a84-46b9-9bb7-3bcf795e0154 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:04.889526 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 973/1923] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:34:04.908069 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f3fb2d-acf6-4b50-979a-f8c353a39378 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:04.910241 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-35f3fb2d-acf6-4b50-979a-f8c353a39378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:04.929679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6949d1fb-5621-4c83-ab55-973c0814139a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:04.930886 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949d1fb-5621-4c83-ab55-973c0814139a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:04.931174 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949d1fb-5621-4c83-ab55-973c0814139a admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:04.931454 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949d1fb-5621-4c83-ab55-973c0814139a admin admin] PATH_INFO: `/v3/users/8670c0d680a64e1684e8be44fe005724` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:04.933212 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6949d1fb-5621-4c83-ab55-973c0814139a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8670c0d680a64e1684e8be44fe005724)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:04.935269 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6949d1fb-5621-4c83-ab55-973c0814139a admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:05.038980 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f3fb2d-acf6-4b50-979a-f8c353a39378 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:05.047719 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5138ab-15f0-46d6-a42a-71d867de1759 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:05.049070 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5138ab-15f0-46d6-a42a-71d867de1759 service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:05.049556 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5138ab-15f0-46d6-a42a-71d867de1759 service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:05.050258 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5138ab-15f0-46d6-a42a-71d867de1759 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:05.062852 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5138ab-15f0-46d6-a42a-71d867de1759 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:05.064906 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5138ab-15f0-46d6-a42a-71d867de1759 service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:05.091049 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 974/1924] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:34:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:34:05.216843 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-6949d1fb-5621-4c83-ab55-973c0814139a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8670c0d680a64e1684e8be44fe005724'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:05.228872 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-6949d1fb-5621-4c83-ab55-973c0814139a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8670c0d680a64e1684e8be44fe005724'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:05.248868 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-6949d1fb-5621-4c83-ab55-973c0814139a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8670c0d680a64e1684e8be44fe005724'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:05.249813 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-6949d1fb-5621-4c83-ab55-973c0814139a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8670c0d680a64e1684e8be44fe005724'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:05.256789 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5138ab-15f0-46d6-a42a-71d867de1759 service neutron] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:05.258307 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-5d5138ab-15f0-46d6-a42a-71d867de1759 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:05.276802 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5138ab-15f0-46d6-a42a-71d867de1759 service neutron] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:05.284610 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e0bcb6-765f-4c82-96c7-fad9b08cb067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:05.286187 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0bcb6-765f-4c82-96c7-fad9b08cb067 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:05.286601 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0bcb6-765f-4c82-96c7-fad9b08cb067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:05.286980 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0bcb6-765f-4c82-96c7-fad9b08cb067 admin admin] PATH_INFO: `/v3/projects/8e9256a368054ed5b20d1cf1ca9bfd15` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:05.289188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e0bcb6-765f-4c82-96c7-fad9b08cb067 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e9256a368054ed5b20d1cf1ca9bfd15)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:05.291589 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e0bcb6-765f-4c82-96c7-fad9b08cb067 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:05.377974 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-6949d1fb-5621-4c83-ab55-973c0814139a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8670c0d680a64e1684e8be44fe005724'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:05.396178 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 951/1925] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:34:04 2022] DELETE /identity/v3/users/8670c0d680a64e1684e8be44fe005724 => generated 0 bytes in 746 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:05.401131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb02a6-3f41-4e43-b59c-c0bd26ec3fc5 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:05.401488 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb02a6-3f41-4e43-b59c-c0bd26ec3fc5 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:05.401883 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb02a6-3f41-4e43-b59c-c0bd26ec3fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:05.429837 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-e3cb02a6-3f41-4e43-b59c-c0bd26ec3fc5 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:05.431161 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-e3cb02a6-3f41-4e43-b59c-c0bd26ec3fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:05.468207 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:05.471791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-d0e0bcb6-765f-4c82-96c7-fad9b08cb067 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e9256a368054ed5b20d1cf1ca9bfd15', 'request_id': 'req-d0e0bcb6-765f-4c82-96c7-fad9b08cb067'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:05.479777 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-d0e0bcb6-765f-4c82-96c7-fad9b08cb067 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e9256a368054ed5b20d1cf1ca9bfd15', 'request_id': 'req-d0e0bcb6-765f-4c82-96c7-fad9b08cb067'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:05.489144 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-d0e0bcb6-765f-4c82-96c7-fad9b08cb067 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e9256a368054ed5b20d1cf1ca9bfd15', 'request_id': 'req-d0e0bcb6-765f-4c82-96c7-fad9b08cb067'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:05.551437 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 952/1926] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:34:05.583766 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb02a6-3f41-4e43-b59c-c0bd26ec3fc5 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:05.587669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-e3cb02a6-3f41-4e43-b59c-c0bd26ec3fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:05.588189 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-d0e0bcb6-765f-4c82-96c7-fad9b08cb067 admin admin] The token cache is being invalidate because project 8e9256a368054ed5b20d1cf1ca9bfd15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:34:05.588585 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-d0e0bcb6-765f-4c82-96c7-fad9b08cb067 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:05.589506 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-d0e0bcb6-765f-4c82-96c7-fad9b08cb067 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:05.592018 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 975/1927] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:34:05 2022] DELETE /identity/v3/projects/8e9256a368054ed5b20d1cf1ca9bfd15 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:34:05.600440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d366c1-75d0-414d-a2d8-e3110007a72d None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:05.600440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d366c1-75d0-414d-a2d8-e3110007a72d None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:05.600440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d366c1-75d0-414d-a2d8-e3110007a72d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:05.646047 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-c2d366c1-75d0-414d-a2d8-e3110007a72d None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:05.654284 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-c2d366c1-75d0-414d-a2d8-e3110007a72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:05.678513 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb02a6-3f41-4e43-b59c-c0bd26ec3fc5 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:05.756688 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b968e1e2-44d3-4228-be20-f951802b0719 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:05.757600 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b968e1e2-44d3-4228-be20-f951802b0719 service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:05.757945 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b968e1e2-44d3-4228-be20-f951802b0719 service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:05.759775 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b968e1e2-44d3-4228-be20-f951802b0719 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:05.766174 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:05.776293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-b968e1e2-44d3-4228-be20-f951802b0719 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:05.784047 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 976/1928] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:05.785323 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d366c1-75d0-414d-a2d8-e3110007a72d None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:05.786762 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-c2d366c1-75d0-414d-a2d8-e3110007a72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:05.794945 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b968e1e2-44d3-4228-be20-f951802b0719 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:05.796819 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b968e1e2-44d3-4228-be20-f951802b0719 service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:05.811703 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d366c1-75d0-414d-a2d8-e3110007a72d None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:05.813755 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 953/1929] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:34:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:34:05.815481 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d3b80-446c-46d6-8c05-dcf9e2fa60b9 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:05.815803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d3b80-446c-46d6-8c05-dcf9e2fa60b9 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:05.817312 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d3b80-446c-46d6-8c05-dcf9e2fa60b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:05.817725 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd69d7ef-5c6b-4335-ba88-64232d2c9346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:05.818791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd69d7ef-5c6b-4335-ba88-64232d2c9346 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:05.819353 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd69d7ef-5c6b-4335-ba88-64232d2c9346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:05.819954 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd69d7ef-5c6b-4335-ba88-64232d2c9346 admin admin] PATH_INFO: `/v3/projects/4b7b566a34fb4a8e8c186e87d351130c` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:05.821212 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd69d7ef-5c6b-4335-ba88-64232d2c9346 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b7b566a34fb4a8e8c186e87d351130c)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:05.823297 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd69d7ef-5c6b-4335-ba88-64232d2c9346 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:05.842046 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-839d3b80-446c-46d6-8c05-dcf9e2fa60b9 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:05.843540 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-839d3b80-446c-46d6-8c05-dcf9e2fa60b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:05.853585 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 954/1930] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:05.858113 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d3b80-446c-46d6-8c05-dcf9e2fa60b9 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:05.874029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d3b80-446c-46d6-8c05-dcf9e2fa60b9 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:05.882006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d5dab8-82b4-4e89-8661-e64f7fbbea2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:05.883006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5dab8-82b4-4e89-8661-e64f7fbbea2a admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:05.883498 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5dab8-82b4-4e89-8661-e64f7fbbea2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:05.883996 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5dab8-82b4-4e89-8661-e64f7fbbea2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:05.885898 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d5dab8-82b4-4e89-8661-e64f7fbbea2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:34:05.886717 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d5dab8-82b4-4e89-8661-e64f7fbbea2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:05.890164 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d5dab8-82b4-4e89-8661-e64f7fbbea2a admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:05.908164 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 955/1931] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:34:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:34:05.912670 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d5dab8-82b4-4e89-8661-e64f7fbbea2a admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:05.949408 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d5dab8-82b4-4e89-8661-e64f7fbbea2a admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:05.959156 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-fd69d7ef-5c6b-4335-ba88-64232d2c9346 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b7b566a34fb4a8e8c186e87d351130c', 'request_id': 'req-fd69d7ef-5c6b-4335-ba88-64232d2c9346'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:05.979147 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-fd69d7ef-5c6b-4335-ba88-64232d2c9346 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b7b566a34fb4a8e8c186e87d351130c', 'request_id': 'req-fd69d7ef-5c6b-4335-ba88-64232d2c9346'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:05.999211 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-fd69d7ef-5c6b-4335-ba88-64232d2c9346 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b7b566a34fb4a8e8c186e87d351130c', 'request_id': 'req-fd69d7ef-5c6b-4335-ba88-64232d2c9346'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:06.065203 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc06a2-7935-4fc4-b7f3-96fca33cc19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:06.065625 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-fd69d7ef-5c6b-4335-ba88-64232d2c9346 admin admin] The token cache is being invalidate because project 4b7b566a34fb4a8e8c186e87d351130c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:34:06.066155 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-fd69d7ef-5c6b-4335-ba88-64232d2c9346 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:06.067523 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-fd69d7ef-5c6b-4335-ba88-64232d2c9346 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:06.067970 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc06a2-7935-4fc4-b7f3-96fca33cc19b admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:06.069033 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc06a2-7935-4fc4-b7f3-96fca33cc19b admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:06.069651 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc06a2-7935-4fc4-b7f3-96fca33cc19b admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:06.070473 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 977/1932] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:34:05 2022] DELETE /identity/v3/projects/4b7b566a34fb4a8e8c186e87d351130c => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:06.071983 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebc06a2-7935-4fc4-b7f3-96fca33cc19b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:06.076587 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebc06a2-7935-4fc4-b7f3-96fca33cc19b admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:06.079798 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd69d7ef-5c6b-4335-ba88-64232d2c9346 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:06.084320 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-fd69d7ef-5c6b-4335-ba88-64232d2c9346 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:06.134893 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd69d7ef-5c6b-4335-ba88-64232d2c9346 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:06.150433 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1def8b-13a9-4c92-b185-b1e323b43c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:06.151323 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1def8b-13a9-4c92-b185-b1e323b43c58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:06.153072 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1def8b-13a9-4c92-b185-b1e323b43c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:06.153350 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1def8b-13a9-4c92-b185-b1e323b43c58 admin admin] PATH_INFO: `/v3/projects/2ca25ffa9f2b43a19a3551000c4594ef` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:06.155448 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1def8b-13a9-4c92-b185-b1e323b43c58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ca25ffa9f2b43a19a3551000c4594ef)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:06.158061 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1def8b-13a9-4c92-b185-b1e323b43c58 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:06.206395 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 956/1933] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:34:05 2022] POST /identity/v3/projects => generated 377 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:34:06.207794 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064353e-1f67-4bd3-94e9-f9b749e5fd04 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:06.208091 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064353e-1f67-4bd3-94e9-f9b749e5fd04 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:06.208371 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064353e-1f67-4bd3-94e9-f9b749e5fd04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:06.242099 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-9064353e-1f67-4bd3-94e9-f9b749e5fd04 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:06.244253 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-9064353e-1f67-4bd3-94e9-f9b749e5fd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:06.321419 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-6e1def8b-13a9-4c92-b185-b1e323b43c58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ca25ffa9f2b43a19a3551000c4594ef', 'request_id': 'req-6e1def8b-13a9-4c92-b185-b1e323b43c58'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:06.323546 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:06.340490 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-6e1def8b-13a9-4c92-b185-b1e323b43c58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ca25ffa9f2b43a19a3551000c4594ef', 'request_id': 'req-6e1def8b-13a9-4c92-b185-b1e323b43c58'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:06.349671 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-6e1def8b-13a9-4c92-b185-b1e323b43c58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ca25ffa9f2b43a19a3551000c4594ef', 'request_id': 'req-6e1def8b-13a9-4c92-b185-b1e323b43c58'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:06.398820 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 957/1934] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:06.398820 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9064353e-1f67-4bd3-94e9-f9b749e5fd04 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:06.398820 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-9064353e-1f67-4bd3-94e9-f9b749e5fd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:06.405605 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9064353e-1f67-4bd3-94e9-f9b749e5fd04 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:06.475633 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6ead4a-50cc-43f1-b555-90ba60db82ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:06.476506 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ead4a-50cc-43f1-b555-90ba60db82ce admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:06.476885 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ead4a-50cc-43f1-b555-90ba60db82ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:06.477254 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ead4a-50cc-43f1-b555-90ba60db82ce admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:06.478176 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6ead4a-50cc-43f1-b555-90ba60db82ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:06.480733 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6ead4a-50cc-43f1-b555-90ba60db82ce admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:06.877480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-6e1def8b-13a9-4c92-b185-b1e323b43c58 admin admin] The token cache is being invalidate because project 2ca25ffa9f2b43a19a3551000c4594ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:34:06.891896 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-6e1def8b-13a9-4c92-b185-b1e323b43c58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:06.891896 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-6e1def8b-13a9-4c92-b185-b1e323b43c58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:06.891896 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 978/1935] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:34:06 2022] DELETE /identity/v3/projects/2ca25ffa9f2b43a19a3551000c4594ef => generated 0 bytes in 809 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:06.891896 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ffaa4c-b386-4525-8242-f25a34f617ef None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:06.891896 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ffaa4c-b386-4525-8242-f25a34f617ef None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:06.891896 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ffaa4c-b386-4525-8242-f25a34f617ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:06.901530 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-d6ffaa4c-b386-4525-8242-f25a34f617ef None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:06.902740 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-d6ffaa4c-b386-4525-8242-f25a34f617ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:06.911304 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 979/1936] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:06.912120 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ffaa4c-b386-4525-8242-f25a34f617ef None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:06.916560 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-d6ffaa4c-b386-4525-8242-f25a34f617ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:06.928105 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ffaa4c-b386-4525-8242-f25a34f617ef None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:06.955628 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2233eb3-5fc0-43ff-8a98-9411c4ce4968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:06.955628 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2233eb3-5fc0-43ff-8a98-9411c4ce4968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:06.955628 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2233eb3-5fc0-43ff-8a98-9411c4ce4968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:06.955628 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2233eb3-5fc0-43ff-8a98-9411c4ce4968 admin admin] PATH_INFO: `/v3/users/d1d11bd791734fc4a457321d180e1c79` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:06.956194 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2233eb3-5fc0-43ff-8a98-9411c4ce4968 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1d11bd791734fc4a457321d180e1c79)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:06.956194 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2233eb3-5fc0-43ff-8a98-9411c4ce4968 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:07.006337 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:07.017578 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 958/1937] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:34:06 2022] POST /identity/v3/users => generated 289 bytes in 628 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:34:07.017578 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6ead4a-50cc-43f1-b555-90ba60db82ce admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:07.029579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6ead4a-50cc-43f1-b555-90ba60db82ce admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:07.034120 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963ce7b-1e9e-4b70-b2e2-6844c40b50a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:07.034776 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963ce7b-1e9e-4b70-b2e2-6844c40b50a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:07.035055 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963ce7b-1e9e-4b70-b2e2-6844c40b50a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:07.035326 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963ce7b-1e9e-4b70-b2e2-6844c40b50a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:07.036088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b963ce7b-1e9e-4b70-b2e2-6844c40b50a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:34:07.036392 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b963ce7b-1e9e-4b70-b2e2-6844c40b50a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:07.037981 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b963ce7b-1e9e-4b70-b2e2-6844c40b50a0 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:07.050506 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 959/1938] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:34:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:34:07.051367 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963ce7b-1e9e-4b70-b2e2-6844c40b50a0 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:07.053095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-b963ce7b-1e9e-4b70-b2e2-6844c40b50a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:07.067135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963ce7b-1e9e-4b70-b2e2-6844c40b50a0 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:07.072776 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc4e5f-9ca7-4f83-b4ce-ff8ea232663d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:07.077550 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc4e5f-9ca7-4f83-b4ce-ff8ea232663d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:07.077550 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc4e5f-9ca7-4f83-b4ce-ff8ea232663d admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:07.077550 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc4e5f-9ca7-4f83-b4ce-ff8ea232663d admin admin] PATH_INFO: `/v3/projects/82288fd065744001b46a7b4cadc9b26c` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:07.077550 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc4e5f-9ca7-4f83-b4ce-ff8ea232663d admin admin] RBAC: Authorizing `identity:delete_project(project_id=82288fd065744001b46a7b4cadc9b26c)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:07.078049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc4e5f-9ca7-4f83-b4ce-ff8ea232663d admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:07.199362 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-befc4e5f-9ca7-4f83-b4ce-ff8ea232663d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82288fd065744001b46a7b4cadc9b26c', 'request_id': 'req-befc4e5f-9ca7-4f83-b4ce-ff8ea232663d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:07.206032 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-befc4e5f-9ca7-4f83-b4ce-ff8ea232663d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82288fd065744001b46a7b4cadc9b26c', 'request_id': 'req-befc4e5f-9ca7-4f83-b4ce-ff8ea232663d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:07.216011 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-befc4e5f-9ca7-4f83-b4ce-ff8ea232663d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82288fd065744001b46a7b4cadc9b26c', 'request_id': 'req-befc4e5f-9ca7-4f83-b4ce-ff8ea232663d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:07.335856 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-befc4e5f-9ca7-4f83-b4ce-ff8ea232663d admin admin] The token cache is being invalidate because project 82288fd065744001b46a7b4cadc9b26c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:34:07.338749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-befc4e5f-9ca7-4f83-b4ce-ff8ea232663d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:07.338749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-befc4e5f-9ca7-4f83-b4ce-ff8ea232663d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:07.340248 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 960/1939] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:34:07 2022] DELETE /identity/v3/projects/82288fd065744001b46a7b4cadc9b26c => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:07.341333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc4e5f-9ca7-4f83-b4ce-ff8ea232663d admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:07.343733 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-befc4e5f-9ca7-4f83-b4ce-ff8ea232663d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:07.350250 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-e2233eb3-5fc0-43ff-8a98-9411c4ce4968 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1d11bd791734fc4a457321d180e1c79'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:07.350964 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-e2233eb3-5fc0-43ff-8a98-9411c4ce4968 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1d11bd791734fc4a457321d180e1c79'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:07.360393 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-e2233eb3-5fc0-43ff-8a98-9411c4ce4968 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1d11bd791734fc4a457321d180e1c79'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:07.361319 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-e2233eb3-5fc0-43ff-8a98-9411c4ce4968 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1d11bd791734fc4a457321d180e1c79'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:07.385372 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-e2233eb3-5fc0-43ff-8a98-9411c4ce4968 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1d11bd791734fc4a457321d180e1c79'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:07.420639 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc4e5f-9ca7-4f83-b4ce-ff8ea232663d admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:07.425492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9811dc-28a0-49e3-9fed-4247576eb264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:07.426256 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9811dc-28a0-49e3-9fed-4247576eb264 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:07.426598 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9811dc-28a0-49e3-9fed-4247576eb264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:07.426876 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9811dc-28a0-49e3-9fed-4247576eb264 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:07.427633 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9811dc-28a0-49e3-9fed-4247576eb264 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:07.429228 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9811dc-28a0-49e3-9fed-4247576eb264 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:07.440986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 961/1940] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:34:07.442112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9811dc-28a0-49e3-9fed-4247576eb264 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:07.444275 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-3f9811dc-28a0-49e3-9fed-4247576eb264 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:07.457608 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9811dc-28a0-49e3-9fed-4247576eb264 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:07.462450 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7b7b09-9613-4acf-a546-0e122dfb089f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:07.463118 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b7b09-9613-4acf-a546-0e122dfb089f admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:07.463399 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b7b09-9613-4acf-a546-0e122dfb089f admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:07.463706 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b7b09-9613-4acf-a546-0e122dfb089f admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:07.464409 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7b7b09-9613-4acf-a546-0e122dfb089f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:07.466357 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7b7b09-9613-4acf-a546-0e122dfb089f admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:07.470467 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 980/1941] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:34:06 2022] DELETE /identity/v3/users/d1d11bd791734fc4a457321d180e1c79 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:07.470467 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2233eb3-5fc0-43ff-8a98-9411c4ce4968 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:07.473773 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-e2233eb3-5fc0-43ff-8a98-9411c4ce4968 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:07.551933 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2233eb3-5fc0-43ff-8a98-9411c4ce4968 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:07.557255 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3fb50-23d4-45e8-9b50-93739afddc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:07.557939 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3fb50-23d4-45e8-9b50-93739afddc40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:07.558218 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3fb50-23d4-45e8-9b50-93739afddc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:07.558648 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3fb50-23d4-45e8-9b50-93739afddc40 admin admin] PATH_INFO: `/v3/users/2ef3a193faab46f68e419eece886fd23` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:07.560465 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd3fb50-23d4-45e8-9b50-93739afddc40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ef3a193faab46f68e419eece886fd23)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:07.562398 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd3fb50-23d4-45e8-9b50-93739afddc40 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:07.581114 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 962/1942] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:34:07 2022] POST /identity/v3/projects => generated 369 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:34:07.581114 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7b7b09-9613-4acf-a546-0e122dfb089f admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:07.582689 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-7a7b7b09-9613-4acf-a546-0e122dfb089f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:07.653731 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7b7b09-9613-4acf-a546-0e122dfb089f admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:07.658486 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c031609a-ca53-44c5-bbac-4489b983f558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:07.659157 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031609a-ca53-44c5-bbac-4489b983f558 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:07.659441 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031609a-ca53-44c5-bbac-4489b983f558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:07.659769 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031609a-ca53-44c5-bbac-4489b983f558 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:07.660506 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c031609a-ca53-44c5-bbac-4489b983f558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:07.662248 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c031609a-ca53-44c5-bbac-4489b983f558 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:07.677808 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 963/1943] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:34:07.677808 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c031609a-ca53-44c5-bbac-4489b983f558 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:07.689318 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c031609a-ca53-44c5-bbac-4489b983f558 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:07.694684 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110370c2-7bbe-4173-8bc9-2944287a0f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:07.695467 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110370c2-7bbe-4173-8bc9-2944287a0f81 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:07.695851 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110370c2-7bbe-4173-8bc9-2944287a0f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:07.696362 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110370c2-7bbe-4173-8bc9-2944287a0f81 admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:07.697632 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110370c2-7bbe-4173-8bc9-2944287a0f81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:07.700122 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110370c2-7bbe-4173-8bc9-2944287a0f81 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:07.714117 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:07.730293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-7bd3fb50-23d4-45e8-9b50-93739afddc40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ef3a193faab46f68e419eece886fd23'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:07.731246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-7bd3fb50-23d4-45e8-9b50-93739afddc40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ef3a193faab46f68e419eece886fd23'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:07.739765 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-7bd3fb50-23d4-45e8-9b50-93739afddc40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ef3a193faab46f68e419eece886fd23'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:07.744905 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-7bd3fb50-23d4-45e8-9b50-93739afddc40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ef3a193faab46f68e419eece886fd23'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:07.754460 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-7bd3fb50-23d4-45e8-9b50-93739afddc40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ef3a193faab46f68e419eece886fd23'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:07.827777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 964/1944] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:34:07 2022] POST /identity/v3/users => generated 285 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:34:07.828678 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110370c2-7bbe-4173-8bc9-2944287a0f81 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:07.830335 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-110370c2-7bbe-4173-8bc9-2944287a0f81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:07.835938 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 981/1945] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:34:07 2022] DELETE /identity/v3/users/2ef3a193faab46f68e419eece886fd23 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:07.837075 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3fb50-23d4-45e8-9b50-93739afddc40 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:07.844125 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-7bd3fb50-23d4-45e8-9b50-93739afddc40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:07.848893 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110370c2-7bbe-4173-8bc9-2944287a0f81 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:07.850278 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:07.932597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3fb50-23d4-45e8-9b50-93739afddc40 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:07.933514 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d77ab8-b944-47e2-a41c-966cc946c870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:07.934420 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d77ab8-b944-47e2-a41c-966cc946c870 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:07.934703 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d77ab8-b944-47e2-a41c-966cc946c870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:07.935011 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d77ab8-b944-47e2-a41c-966cc946c870 admin admin] PATH_INFO: `/v3/projects/04501c20d4f74c39873f862496f17c68/users/8f5c0d17b2354654aa87c2b656709952/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:07.938043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea855fa-c99e-477f-b373-2e1d4989cb57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:07.939036 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea855fa-c99e-477f-b373-2e1d4989cb57 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:07.939036 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea855fa-c99e-477f-b373-2e1d4989cb57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:07.939320 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea855fa-c99e-477f-b373-2e1d4989cb57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:07.940132 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea855fa-c99e-477f-b373-2e1d4989cb57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:07.941848 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea855fa-c99e-477f-b373-2e1d4989cb57 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:07.956639 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 982/1946] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:34:07.956639 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea855fa-c99e-477f-b373-2e1d4989cb57 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:07.957324 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-8ea855fa-c99e-477f-b373-2e1d4989cb57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:07.966827 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d77ab8-b944-47e2-a41c-966cc946c870 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04501c20d4f74c39873f862496f17c68, user_id=8f5c0d17b2354654aa87c2b656709952, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:07.969486 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d77ab8-b944-47e2-a41c-966cc946c870 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:07.971608 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea855fa-c99e-477f-b373-2e1d4989cb57 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:07.977056 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a0edfb-0a27-4e63-b055-0750107564c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:07.980857 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0edfb-0a27-4e63-b055-0750107564c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:07.980857 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0edfb-0a27-4e63-b055-0750107564c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:07.980857 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0edfb-0a27-4e63-b055-0750107564c6 admin admin] PATH_INFO: `/v3/projects/252d5b8ce105419cbcc31870cd0e0e96` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:07.980857 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a0edfb-0a27-4e63-b055-0750107564c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=252d5b8ce105419cbcc31870cd0e0e96)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:07.982648 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a0edfb-0a27-4e63-b055-0750107564c6 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:08.070058 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 965/1947] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:34:07 2022] PUT /identity/v3/projects/04501c20d4f74c39873f862496f17c68/users/8f5c0d17b2354654aa87c2b656709952/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:08.070965 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d77ab8-b944-47e2-a41c-966cc946c870 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:08.088455 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-b1a0edfb-0a27-4e63-b055-0750107564c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '252d5b8ce105419cbcc31870cd0e0e96', 'request_id': 'req-b1a0edfb-0a27-4e63-b055-0750107564c6'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:08.095315 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-b1a0edfb-0a27-4e63-b055-0750107564c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '252d5b8ce105419cbcc31870cd0e0e96', 'request_id': 'req-b1a0edfb-0a27-4e63-b055-0750107564c6'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:08.102748 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-b1a0edfb-0a27-4e63-b055-0750107564c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '252d5b8ce105419cbcc31870cd0e0e96', 'request_id': 'req-b1a0edfb-0a27-4e63-b055-0750107564c6'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:08.149006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d77ab8-b944-47e2-a41c-966cc946c870 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:08.183255 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-b1a0edfb-0a27-4e63-b055-0750107564c6 admin admin] The token cache is being invalidate because project 252d5b8ce105419cbcc31870cd0e0e96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:34:08.183615 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-b1a0edfb-0a27-4e63-b055-0750107564c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:08.184404 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-b1a0edfb-0a27-4e63-b055-0750107564c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:08.186327 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 983/1948] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:34:07 2022] DELETE /identity/v3/projects/252d5b8ce105419cbcc31870cd0e0e96 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:08.187726 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdccd769-1a8b-44f6-8a45-bdfb3671bd7f None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:08.188013 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdccd769-1a8b-44f6-8a45-bdfb3671bd7f None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:08.188287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdccd769-1a8b-44f6-8a45-bdfb3671bd7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:08.194180 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:08.216850 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-fdccd769-1a8b-44f6-8a45-bdfb3671bd7f None None] MFA Rules not processed for user `8f5c0d17b2354654aa87c2b656709952`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:08.217734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e2d4e-f009-47a1-b595-4ad1c42fb1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:08.218050 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-fdccd769-1a8b-44f6-8a45-bdfb3671bd7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:08.218596 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e2d4e-f009-47a1-b595-4ad1c42fb1a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:08.218882 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e2d4e-f009-47a1-b595-4ad1c42fb1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:08.219163 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e2d4e-f009-47a1-b595-4ad1c42fb1a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:08.219942 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e2d4e-f009-47a1-b595-4ad1c42fb1a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:08.221576 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e2d4e-f009-47a1-b595-4ad1c42fb1a5 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:08.233534 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 966/1949] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:34:08.234877 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e31d18-70da-40e8-9ade-8fe2c3f2f470 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:08.235164 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e31d18-70da-40e8-9ade-8fe2c3f2f470 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:08.236137 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e31d18-70da-40e8-9ade-8fe2c3f2f470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:08.255281 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-24e31d18-70da-40e8-9ade-8fe2c3f2f470 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:08.256693 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-24e31d18-70da-40e8-9ade-8fe2c3f2f470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:08.265644 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 967/1950] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:08.268463 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e31d18-70da-40e8-9ade-8fe2c3f2f470 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:08.268463 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-24e31d18-70da-40e8-9ade-8fe2c3f2f470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:08.280373 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e31d18-70da-40e8-9ade-8fe2c3f2f470 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:08.285687 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f726c-1386-481f-89c9-c99ed7bea723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:08.286571 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f726c-1386-481f-89c9-c99ed7bea723 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:08.286890 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f726c-1386-481f-89c9-c99ed7bea723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:08.287017 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 984/1951] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:08.287164 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f726c-1386-481f-89c9-c99ed7bea723 admin admin] PATH_INFO: `/v3/projects/b889917fc9c04bb8b90727f9c83acd82/users/a6cbf2a2de3d4021aa890300765d2ded/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:08.288192 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd4741-8ff8-4238-ad95-a52d3525f4ae None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:08.288484 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd4741-8ff8-4238-ad95-a52d3525f4ae None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:08.288761 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd4741-8ff8-4238-ad95-a52d3525f4ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:08.304515 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f726c-1386-481f-89c9-c99ed7bea723 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b889917fc9c04bb8b90727f9c83acd82, user_id=a6cbf2a2de3d4021aa890300765d2ded, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:08.308256 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f726c-1386-481f-89c9-c99ed7bea723 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:08.310502 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:08.317451 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-24bd4741-8ff8-4238-ad95-a52d3525f4ae None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:08.318627 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-24bd4741-8ff8-4238-ad95-a52d3525f4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:08.328322 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 985/1952] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:08.329223 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd4741-8ff8-4238-ad95-a52d3525f4ae None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:08.330524 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-24bd4741-8ff8-4238-ad95-a52d3525f4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:08.387589 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd4741-8ff8-4238-ad95-a52d3525f4ae None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:08.391984 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e079c846-55d8-40dc-9d97-904d946c2b0d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0d29e718f446e2a9fd1b2575a86041', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:08.392634 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079c846-55d8-40dc-9d97-904d946c2b0d service glance] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:08.393083 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079c846-55d8-40dc-9d97-904d946c2b0d service glance] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:08.393520 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079c846-55d8-40dc-9d97-904d946c2b0d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:08.401419 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 968/1953] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:34:08 2022] PUT /identity/v3/projects/b889917fc9c04bb8b90727f9c83acd82/users/a6cbf2a2de3d4021aa890300765d2ded/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:08.402449 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f726c-1386-481f-89c9-c99ed7bea723 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:08.407446 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-b17f726c-1386-481f-89c9-c99ed7bea723 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:08.407618 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e079c846-55d8-40dc-9d97-904d946c2b0d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:08.407618 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e079c846-55d8-40dc-9d97-904d946c2b0d service glance] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:08.417580 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 986/1954] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:34:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:34:08.418488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e079c846-55d8-40dc-9d97-904d946c2b0d service glance] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:08.432213 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e079c846-55d8-40dc-9d97-904d946c2b0d service glance] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:08.490344 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aca3c38-d327-4165-b58d-4d444a2decc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:08.491064 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca3c38-d327-4165-b58d-4d444a2decc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:08.491342 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca3c38-d327-4165-b58d-4d444a2decc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:08.491653 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca3c38-d327-4165-b58d-4d444a2decc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:08.492524 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aca3c38-d327-4165-b58d-4d444a2decc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:34:08.492973 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aca3c38-d327-4165-b58d-4d444a2decc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:08.494516 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f726c-1386-481f-89c9-c99ed7bea723 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:08.494661 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aca3c38-d327-4165-b58d-4d444a2decc4 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:08.501926 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29214728-6759-4879-8465-55fd46e158fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:08.503045 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29214728-6759-4879-8465-55fd46e158fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:08.503335 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29214728-6759-4879-8465-55fd46e158fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:08.503641 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29214728-6759-4879-8465-55fd46e158fb admin admin] PATH_INFO: `/v3/projects/819cabbf14ab4efab66962d1621ad00c` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:08.505003 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29214728-6759-4879-8465-55fd46e158fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=819cabbf14ab4efab66962d1621ad00c)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:08.510597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 987/1955] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:34:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:34:08.510842 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29214728-6759-4879-8465-55fd46e158fb admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:08.510932 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be60a3-e9ef-4fb9-ac52-f37614842cce None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:08.510932 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be60a3-e9ef-4fb9-ac52-f37614842cce None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:08.510932 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be60a3-e9ef-4fb9-ac52-f37614842cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:08.540970 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-21be60a3-e9ef-4fb9-ac52-f37614842cce None None] MFA Rules not processed for user `a6cbf2a2de3d4021aa890300765d2ded`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:08.541898 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-21be60a3-e9ef-4fb9-ac52-f37614842cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:08.562818 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:08.613287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 988/1956] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:08 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:08.614829 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be60a3-e9ef-4fb9-ac52-f37614842cce None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:08.636150 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-29214728-6759-4879-8465-55fd46e158fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '819cabbf14ab4efab66962d1621ad00c', 'request_id': 'req-29214728-6759-4879-8465-55fd46e158fb'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:08.642891 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-29214728-6759-4879-8465-55fd46e158fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '819cabbf14ab4efab66962d1621ad00c', 'request_id': 'req-29214728-6759-4879-8465-55fd46e158fb'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:08.645853 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be60a3-e9ef-4fb9-ac52-f37614842cce None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:08.649425 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-29214728-6759-4879-8465-55fd46e158fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '819cabbf14ab4efab66962d1621ad00c', 'request_id': 'req-29214728-6759-4879-8465-55fd46e158fb'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:08.706324 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f72ebf-ece5-43d9-a332-7b70d58294a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:08.706980 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f72ebf-ece5-43d9-a332-7b70d58294a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:08.707261 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f72ebf-ece5-43d9-a332-7b70d58294a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:08.707536 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f72ebf-ece5-43d9-a332-7b70d58294a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:08.708259 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f72ebf-ece5-43d9-a332-7b70d58294a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:08.710142 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f72ebf-ece5-43d9-a332-7b70d58294a3 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:08.729354 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-29214728-6759-4879-8465-55fd46e158fb admin admin] The token cache is being invalidate because project 819cabbf14ab4efab66962d1621ad00c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:34:08.730054 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-29214728-6759-4879-8465-55fd46e158fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:08.735639 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-29214728-6759-4879-8465-55fd46e158fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:08.735639 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 969/1957] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:34:08 2022] DELETE /identity/v3/projects/819cabbf14ab4efab66962d1621ad00c => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:08.735639 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29214728-6759-4879-8465-55fd46e158fb admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:08.736948 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-29214728-6759-4879-8465-55fd46e158fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:08.792896 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29214728-6759-4879-8465-55fd46e158fb admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:08.796554 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8537ee10-9fde-4cc8-87eb-6f43d421a33f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0d29e718f446e2a9fd1b2575a86041', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:08.797196 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537ee10-9fde-4cc8-87eb-6f43d421a33f service glance] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:08.797478 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537ee10-9fde-4cc8-87eb-6f43d421a33f service glance] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:08.797757 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537ee10-9fde-4cc8-87eb-6f43d421a33f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:08.799523 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-8537ee10-9fde-4cc8-87eb-6f43d421a33f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:08.811065 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 989/1958] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:34:08 2022] POST /identity/v3/projects => generated 385 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:34:08.812679 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029fb778-2a09-4166-a785-83bff0d4f606 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:08.812982 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029fb778-2a09-4166-a785-83bff0d4f606 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:08.813277 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029fb778-2a09-4166-a785-83bff0d4f606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:08.834337 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-029fb778-2a09-4166-a785-83bff0d4f606 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:08.837554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-029fb778-2a09-4166-a785-83bff0d4f606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:08.873901 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8537ee10-9fde-4cc8-87eb-6f43d421a33f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:08.875272 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8537ee10-9fde-4cc8-87eb-6f43d421a33f service glance] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:08.884772 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 970/1959] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:34:08 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:34:08.885646 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8537ee10-9fde-4cc8-87eb-6f43d421a33f service glance] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:08.887090 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-8537ee10-9fde-4cc8-87eb-6f43d421a33f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:08.890571 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:08.900028 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 990/1960] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:08.904331 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8537ee10-9fde-4cc8-87eb-6f43d421a33f service glance] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:08.909076 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b705b-34a5-4456-a908-f10c3aa76d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:08.909749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b705b-34a5-4456-a908-f10c3aa76d4a admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:08.910031 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b705b-34a5-4456-a908-f10c3aa76d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:08.910311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b705b-34a5-4456-a908-f10c3aa76d4a admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:08.911034 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30b705b-34a5-4456-a908-f10c3aa76d4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:08.912309 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e539d66-0e3d-48bc-b5a4-4e6695ac8bea None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:08.912708 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e539d66-0e3d-48bc-b5a4-4e6695ac8bea None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:08.912998 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30b705b-34a5-4456-a908-f10c3aa76d4a admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:08.913278 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e539d66-0e3d-48bc-b5a4-4e6695ac8bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:08.932057 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-5e539d66-0e3d-48bc-b5a4-4e6695ac8bea None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:08.933247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-5e539d66-0e3d-48bc-b5a4-4e6695ac8bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:08.945319 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 991/1961] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:34:08.946206 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e539d66-0e3d-48bc-b5a4-4e6695ac8bea None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:08.958949 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e539d66-0e3d-48bc-b5a4-4e6695ac8bea None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:09.094361 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e3ed4-bf87-4e6b-a146-1c7c4f3a6d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:09.094361 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e3ed4-bf87-4e6b-a146-1c7c4f3a6d34 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:09.094361 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e3ed4-bf87-4e6b-a146-1c7c4f3a6d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:09.094361 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e3ed4-bf87-4e6b-a146-1c7c4f3a6d34 admin admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:09.095065 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e3ed4-bf87-4e6b-a146-1c7c4f3a6d34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:34:09.095065 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e3ed4-bf87-4e6b-a146-1c7c4f3a6d34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:09.095065 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e3ed4-bf87-4e6b-a146-1c7c4f3a6d34 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:09.095065 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 992/1962] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:34:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:34:09.095065 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e3ed4-bf87-4e6b-a146-1c7c4f3a6d34 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:09.095065 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e3ed4-bf87-4e6b-a146-1c7c4f3a6d34 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:09.095891 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feefbec-a487-41db-885d-b32983141387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:09.095891 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feefbec-a487-41db-885d-b32983141387 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:09.095891 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feefbec-a487-41db-885d-b32983141387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:09.095891 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feefbec-a487-41db-885d-b32983141387 admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:09.096591 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feefbec-a487-41db-885d-b32983141387 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:09.096591 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feefbec-a487-41db-885d-b32983141387 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:09.097210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 971/1963] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:34:08 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:34:09.097210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b705b-34a5-4456-a908-f10c3aa76d4a admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:09.097210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b705b-34a5-4456-a908-f10c3aa76d4a admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:09.097210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d9aa5-92b6-415d-ad27-591cfb6b6f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:09.098217 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d9aa5-92b6-415d-ad27-591cfb6b6f83 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:09.098217 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d9aa5-92b6-415d-ad27-591cfb6b6f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:09.098217 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d9aa5-92b6-415d-ad27-591cfb6b6f83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:09.098217 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d9aa5-92b6-415d-ad27-591cfb6b6f83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:09.098217 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d9aa5-92b6-415d-ad27-591cfb6b6f83 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:09.102628 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 972/1964] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:34:09.103684 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d9aa5-92b6-415d-ad27-591cfb6b6f83 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:09.114621 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d9aa5-92b6-415d-ad27-591cfb6b6f83 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:09.119617 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f731cc5-622b-4e31-8b38-81d66208d426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:09.120285 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f731cc5-622b-4e31-8b38-81d66208d426 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:09.120569 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f731cc5-622b-4e31-8b38-81d66208d426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:09.120846 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f731cc5-622b-4e31-8b38-81d66208d426 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:09.121577 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f731cc5-622b-4e31-8b38-81d66208d426 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:09.123834 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f731cc5-622b-4e31-8b38-81d66208d426 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:09.134364 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 973/1965] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:34:09.138622 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f731cc5-622b-4e31-8b38-81d66208d426 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:09.148491 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f731cc5-622b-4e31-8b38-81d66208d426 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:09.153330 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b66f4-c64a-491c-b151-f38bf82f3fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:09.158098 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b66f4-c64a-491c-b151-f38bf82f3fdf admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:09.158098 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b66f4-c64a-491c-b151-f38bf82f3fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:09.158098 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b66f4-c64a-491c-b151-f38bf82f3fdf admin admin] PATH_INFO: `/v3/projects/00696e7c3c974e5391c6059336c87db4/users/15108bc84138464eba55567b5b87979d/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:09.158440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 993/1966] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:34:09 2022] POST /identity/v3/projects => generated 393 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:34:09.171277 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:09.175882 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908b66f4-c64a-491c-b151-f38bf82f3fdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=00696e7c3c974e5391c6059336c87db4, user_id=15108bc84138464eba55567b5b87979d, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:09.176392 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feefbec-a487-41db-885d-b32983141387 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:09.178984 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908b66f4-c64a-491c-b151-f38bf82f3fdf admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:09.224249 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:09.248068 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feefbec-a487-41db-885d-b32983141387 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:09.252777 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55daaf79-6de5-4d0e-b574-3603e9d59252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:09.253440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55daaf79-6de5-4d0e-b574-3603e9d59252 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:09.253721 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55daaf79-6de5-4d0e-b574-3603e9d59252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:09.253994 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55daaf79-6de5-4d0e-b574-3603e9d59252 admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:09.254674 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55daaf79-6de5-4d0e-b574-3603e9d59252 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:09.256657 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55daaf79-6de5-4d0e-b574-3603e9d59252 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:09.275495 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 974/1967] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:34:09 2022] PUT /identity/v3/projects/00696e7c3c974e5391c6059336c87db4/users/15108bc84138464eba55567b5b87979d/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:34:09.286809 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5230d8-62aa-4f3b-81f2-c11e8bd1daa0 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:09.287104 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5230d8-62aa-4f3b-81f2-c11e8bd1daa0 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:09.287428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5230d8-62aa-4f3b-81f2-c11e8bd1daa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:09.314385 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-af5230d8-62aa-4f3b-81f2-c11e8bd1daa0 None None] MFA Rules not processed for user `15108bc84138464eba55567b5b87979d`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:09.315906 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-af5230d8-62aa-4f3b-81f2-c11e8bd1daa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:09.383609 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 994/1968] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:34:09 2022] POST /identity/v3/users => generated 297 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:34:09.387687 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 975/1969] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:34:09.396024 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55daaf79-6de5-4d0e-b574-3603e9d59252 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:09.407058 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55daaf79-6de5-4d0e-b574-3603e9d59252 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:09.440610 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5230d8-62aa-4f3b-81f2-c11e8bd1daa0 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:09.443434 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-af5230d8-62aa-4f3b-81f2-c11e8bd1daa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:09.465777 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec735b-62c7-412d-b551-391ba700d314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:09.466949 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec735b-62c7-412d-b551-391ba700d314 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:09.467235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec735b-62c7-412d-b551-391ba700d314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:09.467516 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec735b-62c7-412d-b551-391ba700d314 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:09.468272 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec735b-62c7-412d-b551-391ba700d314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:09.469932 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec735b-62c7-412d-b551-391ba700d314 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:09.483699 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 995/1970] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:34:09.484819 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec735b-62c7-412d-b551-391ba700d314 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:09.495312 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec735b-62c7-412d-b551-391ba700d314 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:09.505915 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e72809-c4df-4a3c-86b2-5881c8665aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:09.506377 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5230d8-62aa-4f3b-81f2-c11e8bd1daa0 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:09.506645 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e72809-c4df-4a3c-86b2-5881c8665aae admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:09.506934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e72809-c4df-4a3c-86b2-5881c8665aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:09.507851 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e72809-c4df-4a3c-86b2-5881c8665aae admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:09.508755 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e72809-c4df-4a3c-86b2-5881c8665aae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:09.510712 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6a5ae-ac95-4a18-8ae2-d3f12e0007d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:09.511394 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e72809-c4df-4a3c-86b2-5881c8665aae admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:09.511514 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6a5ae-ac95-4a18-8ae2-d3f12e0007d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:09.511867 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6a5ae-ac95-4a18-8ae2-d3f12e0007d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:09.512088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6a5ae-ac95-4a18-8ae2-d3f12e0007d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:09.522451 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af6a5ae-ac95-4a18-8ae2-d3f12e0007d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:09.524719 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af6a5ae-ac95-4a18-8ae2-d3f12e0007d7 service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:09.529580 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 996/1971] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:34:09.533732 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e72809-c4df-4a3c-86b2-5881c8665aae admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:09.540703 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 976/1972] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:34:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 15:34:09.549912 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e72809-c4df-4a3c-86b2-5881c8665aae admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:09.554670 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb8852f-3f0a-490e-b706-bafdb455d4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:09.555669 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8852f-3f0a-490e-b706-bafdb455d4b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:09.561630 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8852f-3f0a-490e-b706-bafdb455d4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:09.561630 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb8852f-3f0a-490e-b706-bafdb455d4b0 admin admin] PATH_INFO: `/v3/projects/6591449803d2415fb5dbc7c05c908a28/users/6b5384e42b5c4fc78b8bcc0ff8eda3df/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:09.570946 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb8852f-3f0a-490e-b706-bafdb455d4b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6591449803d2415fb5dbc7c05c908a28, user_id=6b5384e42b5c4fc78b8bcc0ff8eda3df, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:09.573514 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb8852f-3f0a-490e-b706-bafdb455d4b0 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:09.678276 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 997/1973] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:34:09 2022] PUT /identity/v3/projects/6591449803d2415fb5dbc7c05c908a28/users/6b5384e42b5c4fc78b8bcc0ff8eda3df/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:09.689396 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f000470-238f-4876-a32e-3513949011b0 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:09.689689 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f000470-238f-4876-a32e-3513949011b0 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:09.689991 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f000470-238f-4876-a32e-3513949011b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:09.719963 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-0f000470-238f-4876-a32e-3513949011b0 None None] MFA Rules not processed for user `6b5384e42b5c4fc78b8bcc0ff8eda3df`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:09.721191 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-0f000470-238f-4876-a32e-3513949011b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:09.786769 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 977/1974] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:09.791387 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb8852f-3f0a-490e-b706-bafdb455d4b0 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:09.864633 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb8852f-3f0a-490e-b706-bafdb455d4b0 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:09.870895 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801cc645-1d9e-4191-8916-ae11c28656fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:09.871602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801cc645-1d9e-4191-8916-ae11c28656fe admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:09.871866 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801cc645-1d9e-4191-8916-ae11c28656fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:09.872141 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801cc645-1d9e-4191-8916-ae11c28656fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:09.872849 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801cc645-1d9e-4191-8916-ae11c28656fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:09.877565 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801cc645-1d9e-4191-8916-ae11c28656fe admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:09.968410 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 998/1975] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:34:09 2022] POST /identity/v3/projects => generated 391 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:34:09.972652 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f000470-238f-4876-a32e-3513949011b0 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:09.984595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f000470-238f-4876-a32e-3513949011b0 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:10.017061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:10.055707 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e37077a-e368-46fd-8fe2-3a4229e9350a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:10.056552 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e37077a-e368-46fd-8fe2-3a4229e9350a admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:10.056843 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e37077a-e368-46fd-8fe2-3a4229e9350a admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:10.057115 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e37077a-e368-46fd-8fe2-3a4229e9350a admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:10.057896 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e37077a-e368-46fd-8fe2-3a4229e9350a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:10.060552 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e37077a-e368-46fd-8fe2-3a4229e9350a admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:10.200008 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 978/1976] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:34:09 2022] POST /identity/v3/users => generated 302 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:34:10.208087 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801cc645-1d9e-4191-8916-ae11c28656fe admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:10.231125 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801cc645-1d9e-4191-8916-ae11c28656fe admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:10.240325 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722bd1af-011d-4042-91dd-759fe6f03f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:10.241109 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bd1af-011d-4042-91dd-759fe6f03f30 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:10.241393 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bd1af-011d-4042-91dd-759fe6f03f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:10.241669 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bd1af-011d-4042-91dd-759fe6f03f30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:10.242462 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722bd1af-011d-4042-91dd-759fe6f03f30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:10.248719 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722bd1af-011d-4042-91dd-759fe6f03f30 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:10.254909 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:10.261695 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 999/1977] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:10 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:34:10.266700 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e37077a-e368-46fd-8fe2-3a4229e9350a admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:10.278850 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e37077a-e368-46fd-8fe2-3a4229e9350a admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:10.285453 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c4574-b14b-4967-9923-3e2cb7b0bb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:10.286515 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c4574-b14b-4967-9923-3e2cb7b0bb8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:10.286797 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c4574-b14b-4967-9923-3e2cb7b0bb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:10.287070 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c4574-b14b-4967-9923-3e2cb7b0bb8e admin admin] PATH_INFO: `/v3/projects/be042c49e89443d79c1ba72a108b0504/users/d4b719331f5b42b9a8e8c9cc97df6414/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:10.303202 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128c4574-b14b-4967-9923-3e2cb7b0bb8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=be042c49e89443d79c1ba72a108b0504, user_id=d4b719331f5b42b9a8e8c9cc97df6414, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:10.306268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128c4574-b14b-4967-9923-3e2cb7b0bb8e admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:10.380903 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 979/1978] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:34:10 2022] PUT /identity/v3/projects/be042c49e89443d79c1ba72a108b0504/users/d4b719331f5b42b9a8e8c9cc97df6414/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:10.385010 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722bd1af-011d-4042-91dd-759fe6f03f30 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:10.396603 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722bd1af-011d-4042-91dd-759fe6f03f30 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:10.453816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a4c6b7-b430-4ca6-b92e-0e3d3c7d6cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:10.454501 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4c6b7-b430-4ca6-b92e-0e3d3c7d6cab admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:10.454792 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4c6b7-b430-4ca6-b92e-0e3d3c7d6cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:10.455071 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4c6b7-b430-4ca6-b92e-0e3d3c7d6cab admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:10.455889 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a4c6b7-b430-4ca6-b92e-0e3d3c7d6cab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:10.457534 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a4c6b7-b430-4ca6-b92e-0e3d3c7d6cab admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:10.467548 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1000/1979] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:10 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:34:10.471947 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c4574-b14b-4967-9923-3e2cb7b0bb8e admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:10.484753 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c4574-b14b-4967-9923-3e2cb7b0bb8e admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:10.492400 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c283bce-63aa-4e73-8814-f78028568d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:10.494115 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c283bce-63aa-4e73-8814-f78028568d26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:10.494523 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c283bce-63aa-4e73-8814-f78028568d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:10.494805 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c283bce-63aa-4e73-8814-f78028568d26 admin admin] PATH_INFO: `/v3/projects/be042c49e89443d79c1ba72a108b0504/users/d4b719331f5b42b9a8e8c9cc97df6414/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:10.496882 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c283bce-63aa-4e73-8814-f78028568d26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be042c49e89443d79c1ba72a108b0504, user_id=d4b719331f5b42b9a8e8c9cc97df6414, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:10.500173 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c283bce-63aa-4e73-8814-f78028568d26 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:10.513928 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 980/1980] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:34:10 2022] PUT /identity/v3/projects/be042c49e89443d79c1ba72a108b0504/users/d4b719331f5b42b9a8e8c9cc97df6414/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:10.525283 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d0af8-3b5c-4e53-a322-00f24a33ead8 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:10.525622 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d0af8-3b5c-4e53-a322-00f24a33ead8 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:10.525949 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d0af8-3b5c-4e53-a322-00f24a33ead8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:10.556246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-ca4d0af8-3b5c-4e53-a322-00f24a33ead8 None None] MFA Rules not processed for user `d4b719331f5b42b9a8e8c9cc97df6414`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:10.557479 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-ca4d0af8-3b5c-4e53-a322-00f24a33ead8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:10.657041 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1001/1981] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:10 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:10.667687 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c283bce-63aa-4e73-8814-f78028568d26 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:10.733054 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c283bce-63aa-4e73-8814-f78028568d26 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:10.736602 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ddb77-101c-4fa9-b08b-9f060c358869 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:10.737242 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ddb77-101c-4fa9-b08b-9f060c358869 service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:10.737521 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ddb77-101c-4fa9-b08b-9f060c358869 service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:10.737793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ddb77-101c-4fa9-b08b-9f060c358869 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:10.746748 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490ddb77-101c-4fa9-b08b-9f060c358869 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:10.748859 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490ddb77-101c-4fa9-b08b-9f060c358869 service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:10.758548 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 981/1982] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:34:10 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:34:10.814472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d0af8-3b5c-4e53-a322-00f24a33ead8 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:10.815916 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-ca4d0af8-3b5c-4e53-a322-00f24a33ead8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:10.827877 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d0af8-3b5c-4e53-a322-00f24a33ead8 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:10.831536 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7fae60-136e-445c-b9a0-f4de3cdb35b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:10.832374 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fae60-136e-445c-b9a0-f4de3cdb35b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:10.832654 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fae60-136e-445c-b9a0-f4de3cdb35b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:10.833325 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fae60-136e-445c-b9a0-f4de3cdb35b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:10.899164 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7fae60-136e-445c-b9a0-f4de3cdb35b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:10.900722 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7fae60-136e-445c-b9a0-f4de3cdb35b0 service cinder] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:10.916384 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1002/1983] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:34:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:34:16.931706 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ddb77-101c-4fa9-b08b-9f060c358869 service cinder] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:17.007947 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ddb77-101c-4fa9-b08b-9f060c358869 service cinder] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:17.012968 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac5d78-c20e-4195-92e2-c8559e925e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:17.013845 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac5d78-c20e-4195-92e2-c8559e925e69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:17.014129 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac5d78-c20e-4195-92e2-c8559e925e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:17.014405 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac5d78-c20e-4195-92e2-c8559e925e69 admin admin] PATH_INFO: `/v3/users/6b5384e42b5c4fc78b8bcc0ff8eda3df` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:17.016290 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac5d78-c20e-4195-92e2-c8559e925e69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b5384e42b5c4fc78b8bcc0ff8eda3df)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:17.021452 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac5d78-c20e-4195-92e2-c8559e925e69 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:17.034028 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:17.236984 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-d0ac5d78-c20e-4195-92e2-c8559e925e69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b5384e42b5c4fc78b8bcc0ff8eda3df'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:17.244335 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-d0ac5d78-c20e-4195-92e2-c8559e925e69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b5384e42b5c4fc78b8bcc0ff8eda3df'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:17.258708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-d0ac5d78-c20e-4195-92e2-c8559e925e69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b5384e42b5c4fc78b8bcc0ff8eda3df'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:17.259476 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-d0ac5d78-c20e-4195-92e2-c8559e925e69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b5384e42b5c4fc78b8bcc0ff8eda3df'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:17.369699 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-d0ac5d78-c20e-4195-92e2-c8559e925e69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b5384e42b5c4fc78b8bcc0ff8eda3df'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:17.381646 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 982/1984] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:34:16 2022] DELETE /identity/v3/users/6b5384e42b5c4fc78b8bcc0ff8eda3df => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:17.387653 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3c6d9-8c38-40d6-9553-1f2186a28959 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:17.388142 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3c6d9-8c38-40d6-9553-1f2186a28959 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:17.388319 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3c6d9-8c38-40d6-9553-1f2186a28959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:17.411087 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-eae3c6d9-8c38-40d6-9553-1f2186a28959 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:17.411087 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-eae3c6d9-8c38-40d6-9553-1f2186a28959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:17.426618 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:17.475541 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1003/1985] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:17.486036 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac5d78-c20e-4195-92e2-c8559e925e69 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:17.487419 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-d0ac5d78-c20e-4195-92e2-c8559e925e69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:17.549101 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac5d78-c20e-4195-92e2-c8559e925e69 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:17.552963 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7036ec-33a6-4e22-9618-a11f7712daa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:17.553660 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7036ec-33a6-4e22-9618-a11f7712daa2 service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:17.554115 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7036ec-33a6-4e22-9618-a11f7712daa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:17.554308 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7036ec-33a6-4e22-9618-a11f7712daa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:17.569186 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7036ec-33a6-4e22-9618-a11f7712daa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:17.570737 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7036ec-33a6-4e22-9618-a11f7712daa2 service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:17.584189 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 983/1986] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:34:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:34:18.036705 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae3c6d9-8c38-40d6-9553-1f2186a28959 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:18.038660 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-eae3c6d9-8c38-40d6-9553-1f2186a28959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:18.060962 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae3c6d9-8c38-40d6-9553-1f2186a28959 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:18.063894 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9955ce88-0ccc-4eff-a0bf-13106894840a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:18.064778 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955ce88-0ccc-4eff-a0bf-13106894840a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:18.065094 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955ce88-0ccc-4eff-a0bf-13106894840a admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:18.065373 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955ce88-0ccc-4eff-a0bf-13106894840a admin admin] PATH_INFO: `/v3/projects/6591449803d2415fb5dbc7c05c908a28` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:18.066579 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9955ce88-0ccc-4eff-a0bf-13106894840a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6591449803d2415fb5dbc7c05c908a28)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:18.068543 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9955ce88-0ccc-4eff-a0bf-13106894840a admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:18.194861 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-9955ce88-0ccc-4eff-a0bf-13106894840a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6591449803d2415fb5dbc7c05c908a28', 'request_id': 'req-9955ce88-0ccc-4eff-a0bf-13106894840a'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:18.200756 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-9955ce88-0ccc-4eff-a0bf-13106894840a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6591449803d2415fb5dbc7c05c908a28', 'request_id': 'req-9955ce88-0ccc-4eff-a0bf-13106894840a'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:18.207861 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-9955ce88-0ccc-4eff-a0bf-13106894840a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6591449803d2415fb5dbc7c05c908a28', 'request_id': 'req-9955ce88-0ccc-4eff-a0bf-13106894840a'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:18.291084 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-9955ce88-0ccc-4eff-a0bf-13106894840a admin admin] The token cache is being invalidate because project 6591449803d2415fb5dbc7c05c908a28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:34:18.291276 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-9955ce88-0ccc-4eff-a0bf-13106894840a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:18.291962 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-9955ce88-0ccc-4eff-a0bf-13106894840a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:18.293537 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1004/1987] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:34:18 2022] DELETE /identity/v3/projects/6591449803d2415fb5dbc7c05c908a28 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:18.300147 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7036ec-33a6-4e22-9618-a11f7712daa2 service neutron] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:18.308782 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7036ec-33a6-4e22-9618-a11f7712daa2 service neutron] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:18.369721 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a564a5-7c68-4109-994c-ef4496dc47ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:18.370637 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a564a5-7c68-4109-994c-ef4496dc47ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:18.370926 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a564a5-7c68-4109-994c-ef4496dc47ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:18.371201 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a564a5-7c68-4109-994c-ef4496dc47ce admin admin] PATH_INFO: `/v3/users/d4b719331f5b42b9a8e8c9cc97df6414` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:18.372428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a564a5-7c68-4109-994c-ef4496dc47ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4b719331f5b42b9a8e8c9cc97df6414)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:18.374846 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a564a5-7c68-4109-994c-ef4496dc47ce admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:18.597100 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:18.609975 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-14a564a5-7c68-4109-994c-ef4496dc47ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4b719331f5b42b9a8e8c9cc97df6414'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:18.616984 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-14a564a5-7c68-4109-994c-ef4496dc47ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4b719331f5b42b9a8e8c9cc97df6414'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:18.630180 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-14a564a5-7c68-4109-994c-ef4496dc47ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4b719331f5b42b9a8e8c9cc97df6414'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:18.630880 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-14a564a5-7c68-4109-994c-ef4496dc47ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4b719331f5b42b9a8e8c9cc97df6414'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:18.728429 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-14a564a5-7c68-4109-994c-ef4496dc47ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4b719331f5b42b9a8e8c9cc97df6414'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:18.736547 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 984/1988] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:34:18 2022] DELETE /identity/v3/users/d4b719331f5b42b9a8e8c9cc97df6414 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:19.179526 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9955ce88-0ccc-4eff-a0bf-13106894840a admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:19.181016 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-9955ce88-0ccc-4eff-a0bf-13106894840a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:19.198147 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9955ce88-0ccc-4eff-a0bf-13106894840a admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:19.257053 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41d5fbe-05c3-49f0-8b6d-5f5957d43064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:19.257913 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d5fbe-05c3-49f0-8b6d-5f5957d43064 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:19.258196 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d5fbe-05c3-49f0-8b6d-5f5957d43064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:19.258465 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d5fbe-05c3-49f0-8b6d-5f5957d43064 admin admin] PATH_INFO: `/v3/projects/be042c49e89443d79c1ba72a108b0504` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:19.259662 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41d5fbe-05c3-49f0-8b6d-5f5957d43064 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be042c49e89443d79c1ba72a108b0504)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:19.261564 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41d5fbe-05c3-49f0-8b6d-5f5957d43064 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:19.367309 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:19.370306 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-f41d5fbe-05c3-49f0-8b6d-5f5957d43064 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be042c49e89443d79c1ba72a108b0504', 'request_id': 'req-f41d5fbe-05c3-49f0-8b6d-5f5957d43064'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:19.376836 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-f41d5fbe-05c3-49f0-8b6d-5f5957d43064 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be042c49e89443d79c1ba72a108b0504', 'request_id': 'req-f41d5fbe-05c3-49f0-8b6d-5f5957d43064'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:19.382995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-f41d5fbe-05c3-49f0-8b6d-5f5957d43064 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be042c49e89443d79c1ba72a108b0504', 'request_id': 'req-f41d5fbe-05c3-49f0-8b6d-5f5957d43064'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:19.501005 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-f41d5fbe-05c3-49f0-8b6d-5f5957d43064 admin admin] The token cache is being invalidate because project be042c49e89443d79c1ba72a108b0504 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:34:19.503764 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-f41d5fbe-05c3-49f0-8b6d-5f5957d43064 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:19.503764 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-f41d5fbe-05c3-49f0-8b6d-5f5957d43064 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:19.503764 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1005/1989] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:34:19 2022] DELETE /identity/v3/projects/be042c49e89443d79c1ba72a108b0504 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:19.508287 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ed5cc-1053-4738-a459-04c7aac9fbdc None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:19.508811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ed5cc-1053-4738-a459-04c7aac9fbdc None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:19.508905 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ed5cc-1053-4738-a459-04c7aac9fbdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:19.529021 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-6c1ed5cc-1053-4738-a459-04c7aac9fbdc None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:19.530334 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-6c1ed5cc-1053-4738-a459-04c7aac9fbdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:19.594323 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 985/1990] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:19.602740 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1ba2d-e2db-4493-a39c-867e54124c5e None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:19.603427 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1ba2d-e2db-4493-a39c-867e54124c5e None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:19.608905 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1ba2d-e2db-4493-a39c-867e54124c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:19.626688 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-1bb1ba2d-e2db-4493-a39c-867e54124c5e None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:19.627948 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-1bb1ba2d-e2db-4493-a39c-867e54124c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:19.636389 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1006/1991] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:19.641112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ed5cc-1053-4738-a459-04c7aac9fbdc None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:19.650766 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ed5cc-1053-4738-a459-04c7aac9fbdc None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:19.656268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b22e43-5590-4afb-9bb8-d4e0ae11ff57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:19.656916 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b22e43-5590-4afb-9bb8-d4e0ae11ff57 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:19.657195 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b22e43-5590-4afb-9bb8-d4e0ae11ff57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:19.657468 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b22e43-5590-4afb-9bb8-d4e0ae11ff57 admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:19.658191 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b22e43-5590-4afb-9bb8-d4e0ae11ff57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:34:19.658497 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b22e43-5590-4afb-9bb8-d4e0ae11ff57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:19.660106 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b22e43-5590-4afb-9bb8-d4e0ae11ff57 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:19.672137 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 986/1992] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:34:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:34:19.676325 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb1ba2d-e2db-4493-a39c-867e54124c5e None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:19.686149 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb1ba2d-e2db-4493-a39c-867e54124c5e None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:19.696500 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd9d51-c04b-4db2-86f9-48cc56268c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:19.697937 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd9d51-c04b-4db2-86f9-48cc56268c67 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:19.697937 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd9d51-c04b-4db2-86f9-48cc56268c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:19.698111 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd9d51-c04b-4db2-86f9-48cc56268c67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:19.698974 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fd9d51-c04b-4db2-86f9-48cc56268c67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:19.701563 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fd9d51-c04b-4db2-86f9-48cc56268c67 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:19.792980 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1007/1993] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:34:19 2022] POST /identity/v3/projects => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:34:19.793122 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b22e43-5590-4afb-9bb8-d4e0ae11ff57 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:19.795555 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-18b22e43-5590-4afb-9bb8-d4e0ae11ff57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:19.797804 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd9d51-c04b-4db2-86f9-48cc56268c67 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:19.816713 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b22e43-5590-4afb-9bb8-d4e0ae11ff57 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:19.870051 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd9d51-c04b-4db2-86f9-48cc56268c67 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:19.876547 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5291249-0d6b-4e87-9b69-a906276c0c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:19.877238 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5291249-0d6b-4e87-9b69-a906276c0c7f admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:19.877523 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5291249-0d6b-4e87-9b69-a906276c0c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:19.877801 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5291249-0d6b-4e87-9b69-a906276c0c7f admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:19.878700 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5291249-0d6b-4e87-9b69-a906276c0c7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:19.880684 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5291249-0d6b-4e87-9b69-a906276c0c7f admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:19.881120 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2fc483-207d-4e90-8f5c-5b924c1b9927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:19.882002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2fc483-207d-4e90-8f5c-5b924c1b9927 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:19.882283 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2fc483-207d-4e90-8f5c-5b924c1b9927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:19.882558 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2fc483-207d-4e90-8f5c-5b924c1b9927 admin admin] PATH_INFO: `/v3/users/15108bc84138464eba55567b5b87979d` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:19.883784 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2fc483-207d-4e90-8f5c-5b924c1b9927 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15108bc84138464eba55567b5b87979d)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:19.885697 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2fc483-207d-4e90-8f5c-5b924c1b9927 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:19.978517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1008/1994] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:34:19 2022] POST /identity/v3/users => generated 292 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:34:19.987247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5291249-0d6b-4e87-9b69-a906276c0c7f admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:19.992867 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5291249-0d6b-4e87-9b69-a906276c0c7f admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:19.997473 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d3f2f3-aeca-44bc-82e4-89b3147747ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:19.998091 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d3f2f3-aeca-44bc-82e4-89b3147747ad admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:19.998375 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d3f2f3-aeca-44bc-82e4-89b3147747ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:19.998651 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d3f2f3-aeca-44bc-82e4-89b3147747ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:19.999483 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d3f2f3-aeca-44bc-82e4-89b3147747ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:20.004569 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d3f2f3-aeca-44bc-82e4-89b3147747ad admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:20.022206 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1009/1995] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:34:20.025168 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d3f2f3-aeca-44bc-82e4-89b3147747ad admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:20.036040 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d3f2f3-aeca-44bc-82e4-89b3147747ad admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:20.041153 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce00213-0fad-4c58-a7cd-6e520dd7a234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:20.041830 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce00213-0fad-4c58-a7cd-6e520dd7a234 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:20.042112 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce00213-0fad-4c58-a7cd-6e520dd7a234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:20.042390 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce00213-0fad-4c58-a7cd-6e520dd7a234 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:20.052010 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce00213-0fad-4c58-a7cd-6e520dd7a234 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:20.052010 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce00213-0fad-4c58-a7cd-6e520dd7a234 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:20.056439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1010/1996] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:34:20.060654 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce00213-0fad-4c58-a7cd-6e520dd7a234 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:20.078763 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce00213-0fad-4c58-a7cd-6e520dd7a234 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:20.083514 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a20ef29-cea3-4320-84e5-8f68b8c4ef0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:20.084437 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20ef29-cea3-4320-84e5-8f68b8c4ef0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:20.084718 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20ef29-cea3-4320-84e5-8f68b8c4ef0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:20.085073 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20ef29-cea3-4320-84e5-8f68b8c4ef0a admin admin] PATH_INFO: `/v3/projects/76b130335c1c46a5b923c6b64766e731/users/76169c0183254560b5e61ed41c9e76e8/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:20.100305 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:20.102232 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a20ef29-cea3-4320-84e5-8f68b8c4ef0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=76b130335c1c46a5b923c6b64766e731, user_id=76169c0183254560b5e61ed41c9e76e8, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:20.103643 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a20ef29-cea3-4320-84e5-8f68b8c4ef0a admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:20.107081 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-aa2fc483-207d-4e90-8f5c-5b924c1b9927 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15108bc84138464eba55567b5b87979d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:20.116737 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-aa2fc483-207d-4e90-8f5c-5b924c1b9927 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15108bc84138464eba55567b5b87979d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:20.129352 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-aa2fc483-207d-4e90-8f5c-5b924c1b9927 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15108bc84138464eba55567b5b87979d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:20.130069 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-aa2fc483-207d-4e90-8f5c-5b924c1b9927 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15108bc84138464eba55567b5b87979d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:20.194374 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-aa2fc483-207d-4e90-8f5c-5b924c1b9927 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15108bc84138464eba55567b5b87979d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:20.200583 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1011/1997] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:34:20 2022] PUT /identity/v3/projects/76b130335c1c46a5b923c6b64766e731/users/76169c0183254560b5e61ed41c9e76e8/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:20.201054 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 987/1998] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:34:19 2022] DELETE /identity/v3/users/15108bc84138464eba55567b5b87979d => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:34:20.205065 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a695ab3-0f95-4c94-a9d0-b70e285d9123 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:20.205480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a695ab3-0f95-4c94-a9d0-b70e285d9123 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:20.205820 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a695ab3-0f95-4c94-a9d0-b70e285d9123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:20.210892 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae28860f-bcce-44a0-b205-dcc4872151c1 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:20.211181 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae28860f-bcce-44a0-b205-dcc4872151c1 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:20.211463 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae28860f-bcce-44a0-b205-dcc4872151c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:20.227689 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-6a695ab3-0f95-4c94-a9d0-b70e285d9123 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:20.229478 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-6a695ab3-0f95-4c94-a9d0-b70e285d9123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:20.245967 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-ae28860f-bcce-44a0-b205-dcc4872151c1 None None] MFA Rules not processed for user `76169c0183254560b5e61ed41c9e76e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:20.253312 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-ae28860f-bcce-44a0-b205-dcc4872151c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:20.279718 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:DELETE=4,keystone:SELECT=90,keystone:INSERT=6 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:20.294710 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1012/1999] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:34:20.306267 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a695ab3-0f95-4c94-a9d0-b70e285d9123 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:20.307823 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-6a695ab3-0f95-4c94-a9d0-b70e285d9123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:20.318719 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 988/2000] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:34:20.323208 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae28860f-bcce-44a0-b205-dcc4872151c1 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:20.324543 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-ae28860f-bcce-44a0-b205-dcc4872151c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:20.338256 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae28860f-bcce-44a0-b205-dcc4872151c1 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:20.342797 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c8cd27-34cd-4496-ba9f-430a12685023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:20.343548 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8cd27-34cd-4496-ba9f-430a12685023 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:20.343773 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8cd27-34cd-4496-ba9f-430a12685023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:20.344047 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8cd27-34cd-4496-ba9f-430a12685023 admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:20.346575 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c8cd27-34cd-4496-ba9f-430a12685023 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:20.346575 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c8cd27-34cd-4496-ba9f-430a12685023 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:20.367120 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a695ab3-0f95-4c94-a9d0-b70e285d9123 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:20.371256 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa177a4e-2ea2-4244-a9e4-0275e992bfbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:20.378345 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa177a4e-2ea2-4244-a9e4-0275e992bfbf service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:20.378345 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa177a4e-2ea2-4244-a9e4-0275e992bfbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:20.378345 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa177a4e-2ea2-4244-a9e4-0275e992bfbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:20.388314 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa177a4e-2ea2-4244-a9e4-0275e992bfbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:20.389776 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa177a4e-2ea2-4244-a9e4-0275e992bfbf service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:20.399902 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1013/2001] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:34:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:34:20.433541 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 989/2002] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:34:20 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:34:20.434945 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa177a4e-2ea2-4244-a9e4-0275e992bfbf service neutron] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:20.445184 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa177a4e-2ea2-4244-a9e4-0275e992bfbf service neutron] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:20.503730 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a0653c-cf64-4f43-9869-edc595c4ea25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:20.504489 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0653c-cf64-4f43-9869-edc595c4ea25 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:20.504715 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0653c-cf64-4f43-9869-edc595c4ea25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:20.504992 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0653c-cf64-4f43-9869-edc595c4ea25 admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:20.505702 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a0653c-cf64-4f43-9869-edc595c4ea25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:20.507733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a0653c-cf64-4f43-9869-edc595c4ea25 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:20.622806 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1014/2003] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:34:20 2022] POST /identity/v3/users => generated 298 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:34:20.624050 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c8cd27-34cd-4496-ba9f-430a12685023 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:20.638728 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c8cd27-34cd-4496-ba9f-430a12685023 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:20.648928 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd8330-decb-46db-a8b3-5f68a149f257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:20.649701 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd8330-decb-46db-a8b3-5f68a149f257 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:20.649989 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd8330-decb-46db-a8b3-5f68a149f257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:20.650287 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd8330-decb-46db-a8b3-5f68a149f257 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:20.651006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd8330-decb-46db-a8b3-5f68a149f257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:20.652666 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd8330-decb-46db-a8b3-5f68a149f257 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:20.665557 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 990/2004] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:34:20.669921 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a0653c-cf64-4f43-9869-edc595c4ea25 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:20.681457 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a0653c-cf64-4f43-9869-edc595c4ea25 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:20.686186 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56868ddc-925f-4de5-b762-56f0a830ccf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:20.687147 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56868ddc-925f-4de5-b762-56f0a830ccf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:20.687979 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56868ddc-925f-4de5-b762-56f0a830ccf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:20.688258 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56868ddc-925f-4de5-b762-56f0a830ccf5 admin admin] PATH_INFO: `/v3/projects/dbb46ffaa3eb4e32806f024ccb8c52f6/users/331c93aed8de463984463b0e0ea1786c/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:20.710030 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56868ddc-925f-4de5-b762-56f0a830ccf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbb46ffaa3eb4e32806f024ccb8c52f6, user_id=331c93aed8de463984463b0e0ea1786c, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:20.712759 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56868ddc-925f-4de5-b762-56f0a830ccf5 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:20.790075 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1015/2005] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:34:20 2022] PUT /identity/v3/projects/dbb46ffaa3eb4e32806f024ccb8c52f6/users/331c93aed8de463984463b0e0ea1786c/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:20.794081 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd8330-decb-46db-a8b3-5f68a149f257 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:20.804672 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd8330-decb-46db-a8b3-5f68a149f257 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:20.865008 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf400660-e239-44bd-a644-10c3d9eab4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:20.865687 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf400660-e239-44bd-a644-10c3d9eab4f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:20.865984 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf400660-e239-44bd-a644-10c3d9eab4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:20.866290 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf400660-e239-44bd-a644-10c3d9eab4f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:20.867028 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf400660-e239-44bd-a644-10c3d9eab4f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:20.868842 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf400660-e239-44bd-a644-10c3d9eab4f7 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:20.880195 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 991/2006] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:34:20.884133 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56868ddc-925f-4de5-b762-56f0a830ccf5 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:20.894699 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56868ddc-925f-4de5-b762-56f0a830ccf5 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:20.900516 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f34f5d-df47-4a9d-914f-67469d0a5f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:20.901660 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f34f5d-df47-4a9d-914f-67469d0a5f29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:20.901660 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f34f5d-df47-4a9d-914f-67469d0a5f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:20.902071 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f34f5d-df47-4a9d-914f-67469d0a5f29 admin admin] PATH_INFO: `/v3/projects/dbb46ffaa3eb4e32806f024ccb8c52f6/users/331c93aed8de463984463b0e0ea1786c/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:20.904184 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f34f5d-df47-4a9d-914f-67469d0a5f29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbb46ffaa3eb4e32806f024ccb8c52f6, user_id=331c93aed8de463984463b0e0ea1786c, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:20.904878 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf400660-e239-44bd-a644-10c3d9eab4f7 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:20.907254 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-bf400660-e239-44bd-a644-10c3d9eab4f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:20.907453 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f34f5d-df47-4a9d-914f-67469d0a5f29 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:20.920804 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf400660-e239-44bd-a644-10c3d9eab4f7 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:20.928426 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:20.929843 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1016/2007] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:34:20 2022] PUT /identity/v3/projects/dbb46ffaa3eb4e32806f024ccb8c52f6/users/331c93aed8de463984463b0e0ea1786c/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:20.941634 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c61a06-f41f-43dc-9557-7fbf471f3bdd None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:20.941994 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c61a06-f41f-43dc-9557-7fbf471f3bdd None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:20.942225 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c61a06-f41f-43dc-9557-7fbf471f3bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:20.974350 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-84c61a06-f41f-43dc-9557-7fbf471f3bdd None None] MFA Rules not processed for user `331c93aed8de463984463b0e0ea1786c`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:20.975721 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-84c61a06-f41f-43dc-9557-7fbf471f3bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:20.989241 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d4bc1-cc5c-45a3-8688-0df0b3dd9f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:20.990103 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d4bc1-cc5c-45a3-8688-0df0b3dd9f97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:20.990410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d4bc1-cc5c-45a3-8688-0df0b3dd9f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:20.990681 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d4bc1-cc5c-45a3-8688-0df0b3dd9f97 admin admin] PATH_INFO: `/v3/projects/00696e7c3c974e5391c6059336c87db4` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:20.991910 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422d4bc1-cc5c-45a3-8688-0df0b3dd9f97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00696e7c3c974e5391c6059336c87db4)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:20.993849 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422d4bc1-cc5c-45a3-8688-0df0b3dd9f97 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:21.009847 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:21.050263 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1017/2008] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:20 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:21.061152 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c61a06-f41f-43dc-9557-7fbf471f3bdd None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:21.062663 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-84c61a06-f41f-43dc-9557-7fbf471f3bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:21.124981 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c61a06-f41f-43dc-9557-7fbf471f3bdd None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:21.129477 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1270f6-6128-4054-bd34-d928c52a40ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:21.130218 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1270f6-6128-4054-bd34-d928c52a40ab service cinder] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:21.130525 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1270f6-6128-4054-bd34-d928c52a40ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:21.130774 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1270f6-6128-4054-bd34-d928c52a40ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:21.142967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1270f6-6128-4054-bd34-d928c52a40ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:21.144713 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1270f6-6128-4054-bd34-d928c52a40ab service cinder] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:21.159737 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1018/2009] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:34:21 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 15:34:21.170035 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-422d4bc1-cc5c-45a3-8688-0df0b3dd9f97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00696e7c3c974e5391c6059336c87db4', 'request_id': 'req-422d4bc1-cc5c-45a3-8688-0df0b3dd9f97'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:21.177161 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-422d4bc1-cc5c-45a3-8688-0df0b3dd9f97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00696e7c3c974e5391c6059336c87db4', 'request_id': 'req-422d4bc1-cc5c-45a3-8688-0df0b3dd9f97'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:21.186539 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-422d4bc1-cc5c-45a3-8688-0df0b3dd9f97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00696e7c3c974e5391c6059336c87db4', 'request_id': 'req-422d4bc1-cc5c-45a3-8688-0df0b3dd9f97'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:21.266453 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-422d4bc1-cc5c-45a3-8688-0df0b3dd9f97 admin admin] The token cache is being invalidate because project 00696e7c3c974e5391c6059336c87db4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:34:21.266669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-422d4bc1-cc5c-45a3-8688-0df0b3dd9f97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:21.267315 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-422d4bc1-cc5c-45a3-8688-0df0b3dd9f97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:21.268963 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 992/2010] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:34:20 2022] DELETE /identity/v3/projects/00696e7c3c974e5391c6059336c87db4 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:21.274008 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4395a20-1fe0-4b2d-b111-90e735e311af None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:21.274520 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4395a20-1fe0-4b2d-b111-90e735e311af None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:21.274630 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4395a20-1fe0-4b2d-b111-90e735e311af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:21.302072 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-e4395a20-1fe0-4b2d-b111-90e735e311af None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:21.303232 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-e4395a20-1fe0-4b2d-b111-90e735e311af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:21.306293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d4bc1-cc5c-45a3-8688-0df0b3dd9f97 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:21.307921 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-422d4bc1-cc5c-45a3-8688-0df0b3dd9f97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:21.371750 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1019/2011] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:21.379209 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb7db1-1918-4be7-910d-f31ca6bf24df None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:21.379498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb7db1-1918-4be7-910d-f31ca6bf24df None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:21.379820 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb7db1-1918-4be7-910d-f31ca6bf24df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:21.380793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d4bc1-cc5c-45a3-8688-0df0b3dd9f97 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:21.384694 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe9e7b-27ed-4a36-b0df-f748411b4b90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:21.385349 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe9e7b-27ed-4a36-b0df-f748411b4b90 service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:21.385637 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe9e7b-27ed-4a36-b0df-f748411b4b90 service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:21.385920 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe9e7b-27ed-4a36-b0df-f748411b4b90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:21.387816 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-78fe9e7b-27ed-4a36-b0df-f748411b4b90 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:21.399273 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-8fdb7db1-1918-4be7-910d-f31ca6bf24df None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:21.400637 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-8fdb7db1-1918-4be7-910d-f31ca6bf24df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:21.411309 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1020/2012] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:34:21.415250 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdb7db1-1918-4be7-910d-f31ca6bf24df None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:21.424605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdb7db1-1918-4be7-910d-f31ca6bf24df None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:21.435516 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0572b-2fbe-4dfc-bdbc-1957fb8338ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:21.435817 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0572b-2fbe-4dfc-bdbc-1957fb8338ea admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:21.436009 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0572b-2fbe-4dfc-bdbc-1957fb8338ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:21.436338 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0572b-2fbe-4dfc-bdbc-1957fb8338ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:21.439082 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e0572b-2fbe-4dfc-bdbc-1957fb8338ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:34:21.439394 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e0572b-2fbe-4dfc-bdbc-1957fb8338ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:21.441062 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e0572b-2fbe-4dfc-bdbc-1957fb8338ea admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:21.455607 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1021/2013] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:34:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:34:21.458775 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:21.458978 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0572b-2fbe-4dfc-bdbc-1957fb8338ea admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:21.462781 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe9e7b-27ed-4a36-b0df-f748411b4b90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:21.465115 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe9e7b-27ed-4a36-b0df-f748411b4b90 service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:21.472310 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0572b-2fbe-4dfc-bdbc-1957fb8338ea admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:21.476006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 993/2014] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:34:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 15:34:21.477700 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2684c44d-1eeb-477a-9dc4-f8b4940d2b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:21.478504 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684c44d-1eeb-477a-9dc4-f8b4940d2b6e admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:21.484094 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684c44d-1eeb-477a-9dc4-f8b4940d2b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:21.484094 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684c44d-1eeb-477a-9dc4-f8b4940d2b6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:21.484094 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2684c44d-1eeb-477a-9dc4-f8b4940d2b6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:21.484094 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2684c44d-1eeb-477a-9dc4-f8b4940d2b6e admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:21.573542 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1022/2015] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:34:21 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:34:21.577516 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe9e7b-27ed-4a36-b0df-f748411b4b90 service cinder] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:21.590489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe9e7b-27ed-4a36-b0df-f748411b4b90 service cinder] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:21.673950 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b9cb48-7467-49bd-8569-464c8174049f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:21.674760 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9cb48-7467-49bd-8569-464c8174049f admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:21.675138 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9cb48-7467-49bd-8569-464c8174049f admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:21.675582 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9cb48-7467-49bd-8569-464c8174049f admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:21.676384 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b9cb48-7467-49bd-8569-464c8174049f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:21.678749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b9cb48-7467-49bd-8569-464c8174049f admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:21.766853 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 994/2016] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:34:21 2022] POST /identity/v3/users => generated 287 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:34:21.768157 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2684c44d-1eeb-477a-9dc4-f8b4940d2b6e admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:21.779021 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2684c44d-1eeb-477a-9dc4-f8b4940d2b6e admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:21.793956 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2ad55-fe5d-456d-b7fe-226dfc1e4119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:21.793956 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2ad55-fe5d-456d-b7fe-226dfc1e4119 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:21.793956 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2ad55-fe5d-456d-b7fe-226dfc1e4119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:21.793956 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2ad55-fe5d-456d-b7fe-226dfc1e4119 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:21.794330 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a2ad55-fe5d-456d-b7fe-226dfc1e4119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:21.795419 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a2ad55-fe5d-456d-b7fe-226dfc1e4119 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:21.823127 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1023/2017] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:34:21.831283 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b9cb48-7467-49bd-8569-464c8174049f admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:21.842279 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b9cb48-7467-49bd-8569-464c8174049f admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:21.847104 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2925b-aad4-4714-bf4e-cd82052eb64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:21.847884 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2925b-aad4-4714-bf4e-cd82052eb64b admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:21.848167 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2925b-aad4-4714-bf4e-cd82052eb64b admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:21.848440 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2925b-aad4-4714-bf4e-cd82052eb64b admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:21.849347 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e2925b-aad4-4714-bf4e-cd82052eb64b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:21.850955 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e2925b-aad4-4714-bf4e-cd82052eb64b admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:21.867614 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 995/2018] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:34:21.869341 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2ad55-fe5d-456d-b7fe-226dfc1e4119 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:21.880061 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2ad55-fe5d-456d-b7fe-226dfc1e4119 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:21.884598 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a29c2f-0d88-47bb-9382-c3ee83cf545a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:21.885483 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a29c2f-0d88-47bb-9382-c3ee83cf545a admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:21.885771 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a29c2f-0d88-47bb-9382-c3ee83cf545a admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:21.886044 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a29c2f-0d88-47bb-9382-c3ee83cf545a admin admin] PATH_INFO: `/v3/projects/3c3b09a5c1fe422d8318bf6a1a26c8ee/users/09ead7f054e344c7a965aec56e59dd86/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:21.904378 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a29c2f-0d88-47bb-9382-c3ee83cf545a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c3b09a5c1fe422d8318bf6a1a26c8ee, user_id=09ead7f054e344c7a965aec56e59dd86, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:21.907397 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a29c2f-0d88-47bb-9382-c3ee83cf545a admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:21.913073 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:21.953953 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2925b-aad4-4714-bf4e-cd82052eb64b admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:21.957028 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-d3e2925b-aad4-4714-bf4e-cd82052eb64b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:21.972695 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2925b-aad4-4714-bf4e-cd82052eb64b admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:21.977443 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f32ff-ce96-4c1c-89b1-a56be0b556d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:21.978355 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f32ff-ce96-4c1c-89b1-a56be0b556d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:21.978644 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f32ff-ce96-4c1c-89b1-a56be0b556d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:21.978915 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f32ff-ce96-4c1c-89b1-a56be0b556d4 admin admin] PATH_INFO: `/v3/users/76169c0183254560b5e61ed41c9e76e8` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:21.980153 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65f32ff-ce96-4c1c-89b1-a56be0b556d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76169c0183254560b5e61ed41c9e76e8)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:21.982389 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65f32ff-ce96-4c1c-89b1-a56be0b556d4 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:22.009837 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1024/2019] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:34:21 2022] PUT /identity/v3/projects/3c3b09a5c1fe422d8318bf6a1a26c8ee/users/09ead7f054e344c7a965aec56e59dd86/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:22.021606 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6607b-4920-4128-b7a8-0872bfea1989 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:22.021889 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6607b-4920-4128-b7a8-0872bfea1989 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:22.022167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6607b-4920-4128-b7a8-0872bfea1989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:22.053292 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-01b6607b-4920-4128-b7a8-0872bfea1989 None None] MFA Rules not processed for user `09ead7f054e344c7a965aec56e59dd86`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:22.054498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-01b6607b-4920-4128-b7a8-0872bfea1989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:22.143660 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1025/2020] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:34:22.145283 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b6607b-4920-4128-b7a8-0872bfea1989 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:22.208156 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b6607b-4920-4128-b7a8-0872bfea1989 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:22.211951 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7720e16-3920-4318-bab1-f73d3f8b3516 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:22.212595 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7720e16-3920-4318-bab1-f73d3f8b3516 service cinder] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:22.212871 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7720e16-3920-4318-bab1-f73d3f8b3516 service cinder] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:22.213161 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7720e16-3920-4318-bab1-f73d3f8b3516 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:22.213988 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-c65f32ff-ce96-4c1c-89b1-a56be0b556d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76169c0183254560b5e61ed41c9e76e8'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:22.223002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-c65f32ff-ce96-4c1c-89b1-a56be0b556d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76169c0183254560b5e61ed41c9e76e8'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:22.224982 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7720e16-3920-4318-bab1-f73d3f8b3516 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:22.226449 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7720e16-3920-4318-bab1-f73d3f8b3516 service cinder] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:22.237206 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1026/2021] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:34:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 15:34:22.237471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-c65f32ff-ce96-4c1c-89b1-a56be0b556d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76169c0183254560b5e61ed41c9e76e8'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:22.238165 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-c65f32ff-ce96-4c1c-89b1-a56be0b556d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76169c0183254560b5e61ed41c9e76e8'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:22.300200 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-c65f32ff-ce96-4c1c-89b1-a56be0b556d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76169c0183254560b5e61ed41c9e76e8'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:22.333243 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 996/2022] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:34:21 2022] DELETE /identity/v3/users/76169c0183254560b5e61ed41c9e76e8 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:22.333433 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a91ff0-e3bc-40b3-927d-c35d3bf745a2 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:22.334143 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a91ff0-e3bc-40b3-927d-c35d3bf745a2 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:22.334143 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a91ff0-e3bc-40b3-927d-c35d3bf745a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:22.358352 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-30a91ff0-e3bc-40b3-927d-c35d3bf745a2 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:22.359958 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-30a91ff0-e3bc-40b3-927d-c35d3bf745a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:22.457849 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1027/2023] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:22.467807 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f32ff-ce96-4c1c-89b1-a56be0b556d4 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:22.470000 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-c65f32ff-ce96-4c1c-89b1-a56be0b556d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:22.534826 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:22.545963 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f32ff-ce96-4c1c-89b1-a56be0b556d4 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:22.549851 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912060c-bd9f-420e-abb6-4c857f7dbbee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:22.550595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912060c-bd9f-420e-abb6-4c857f7dbbee service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:22.550886 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912060c-bd9f-420e-abb6-4c857f7dbbee service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:22.551163 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912060c-bd9f-420e-abb6-4c857f7dbbee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:22.562759 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912060c-bd9f-420e-abb6-4c857f7dbbee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:22.564938 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912060c-bd9f-420e-abb6-4c857f7dbbee service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:22.577028 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 997/2024] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:34:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:34:23.542049 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a91ff0-e3bc-40b3-927d-c35d3bf745a2 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:23.543604 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-30a91ff0-e3bc-40b3-927d-c35d3bf745a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:23.560325 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a91ff0-e3bc-40b3-927d-c35d3bf745a2 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:23.566647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406e64e-0719-4c82-abfb-5f5c1fe1350b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:23.567596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406e64e-0719-4c82-abfb-5f5c1fe1350b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:23.568355 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406e64e-0719-4c82-abfb-5f5c1fe1350b admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:23.568355 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406e64e-0719-4c82-abfb-5f5c1fe1350b admin admin] PATH_INFO: `/v3/projects/76b130335c1c46a5b923c6b64766e731` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:23.569741 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5406e64e-0719-4c82-abfb-5f5c1fe1350b admin admin] RBAC: Authorizing `identity:delete_project(project_id=76b130335c1c46a5b923c6b64766e731)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:23.572065 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5406e64e-0719-4c82-abfb-5f5c1fe1350b admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:23.625524 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:23.740333 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-5406e64e-0719-4c82-abfb-5f5c1fe1350b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76b130335c1c46a5b923c6b64766e731', 'request_id': 'req-5406e64e-0719-4c82-abfb-5f5c1fe1350b'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:23.748373 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-5406e64e-0719-4c82-abfb-5f5c1fe1350b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76b130335c1c46a5b923c6b64766e731', 'request_id': 'req-5406e64e-0719-4c82-abfb-5f5c1fe1350b'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:23.757556 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-5406e64e-0719-4c82-abfb-5f5c1fe1350b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76b130335c1c46a5b923c6b64766e731', 'request_id': 'req-5406e64e-0719-4c82-abfb-5f5c1fe1350b'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:23.846048 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-5406e64e-0719-4c82-abfb-5f5c1fe1350b admin admin] The token cache is being invalidate because project 76b130335c1c46a5b923c6b64766e731 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:34:23.846497 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-5406e64e-0719-4c82-abfb-5f5c1fe1350b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:23.847225 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-5406e64e-0719-4c82-abfb-5f5c1fe1350b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:23.853169 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912060c-bd9f-420e-abb6-4c857f7dbbee service neutron] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:23.865131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912060c-bd9f-420e-abb6-4c857f7dbbee service neutron] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:23.865447 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1028/2025] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:34:23 2022] DELETE /identity/v3/projects/76b130335c1c46a5b923c6b64766e731 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:23.942432 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae99954-0cf3-4ffc-b1fb-e6f1db9d3901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:23.943498 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae99954-0cf3-4ffc-b1fb-e6f1db9d3901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:23.943928 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae99954-0cf3-4ffc-b1fb-e6f1db9d3901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:23.944290 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae99954-0cf3-4ffc-b1fb-e6f1db9d3901 admin admin] PATH_INFO: `/v3/users/331c93aed8de463984463b0e0ea1786c` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:23.945632 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae99954-0cf3-4ffc-b1fb-e6f1db9d3901 admin admin] RBAC: Authorizing `identity:delete_user(user_id=331c93aed8de463984463b0e0ea1786c)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:23.947946 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae99954-0cf3-4ffc-b1fb-e6f1db9d3901 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:24.141321 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-4ae99954-0cf3-4ffc-b1fb-e6f1db9d3901 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '331c93aed8de463984463b0e0ea1786c'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:24.150663 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-4ae99954-0cf3-4ffc-b1fb-e6f1db9d3901 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '331c93aed8de463984463b0e0ea1786c'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:24.165978 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-4ae99954-0cf3-4ffc-b1fb-e6f1db9d3901 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '331c93aed8de463984463b0e0ea1786c'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:24.167093 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-4ae99954-0cf3-4ffc-b1fb-e6f1db9d3901 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '331c93aed8de463984463b0e0ea1786c'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:24.257856 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-4ae99954-0cf3-4ffc-b1fb-e6f1db9d3901 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '331c93aed8de463984463b0e0ea1786c'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:24.280056 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 998/2026] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:34:23 2022] DELETE /identity/v3/users/331c93aed8de463984463b0e0ea1786c => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:24.834930 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406e64e-0719-4c82-abfb-5f5c1fe1350b admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:24.837804 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-5406e64e-0719-4c82-abfb-5f5c1fe1350b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:24.852268 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406e64e-0719-4c82-abfb-5f5c1fe1350b admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:24.990014 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2006ab-f96d-467c-bb02-849c34566814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:24.990014 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2006ab-f96d-467c-bb02-849c34566814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:24.990014 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2006ab-f96d-467c-bb02-849c34566814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:24.990014 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2006ab-f96d-467c-bb02-849c34566814 admin admin] PATH_INFO: `/v3/projects/dbb46ffaa3eb4e32806f024ccb8c52f6` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:24.990836 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2006ab-f96d-467c-bb02-849c34566814 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbb46ffaa3eb4e32806f024ccb8c52f6)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:24.994235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2006ab-f96d-467c-bb02-849c34566814 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:25.142715 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-cd2006ab-f96d-467c-bb02-849c34566814 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbb46ffaa3eb4e32806f024ccb8c52f6', 'request_id': 'req-cd2006ab-f96d-467c-bb02-849c34566814'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:25.148949 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-cd2006ab-f96d-467c-bb02-849c34566814 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbb46ffaa3eb4e32806f024ccb8c52f6', 'request_id': 'req-cd2006ab-f96d-467c-bb02-849c34566814'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:25.164090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-cd2006ab-f96d-467c-bb02-849c34566814 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbb46ffaa3eb4e32806f024ccb8c52f6', 'request_id': 'req-cd2006ab-f96d-467c-bb02-849c34566814'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:25.269096 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-cd2006ab-f96d-467c-bb02-849c34566814 admin admin] The token cache is being invalidate because project dbb46ffaa3eb4e32806f024ccb8c52f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:34:25.273284 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-cd2006ab-f96d-467c-bb02-849c34566814 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:25.273284 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-cd2006ab-f96d-467c-bb02-849c34566814 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:25.273284 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1029/2027] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:34:24 2022] DELETE /identity/v3/projects/dbb46ffaa3eb4e32806f024ccb8c52f6 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:25.276878 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fcb84-faf8-4dcc-ae69-79a3bd572777 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:25.277994 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fcb84-faf8-4dcc-ae69-79a3bd572777 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:25.278495 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fcb84-faf8-4dcc-ae69-79a3bd572777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:25.303417 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-4b0fcb84-faf8-4dcc-ae69-79a3bd572777 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:25.304987 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-4b0fcb84-faf8-4dcc-ae69-79a3bd572777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:25.404485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 999/2028] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:25.407843 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f08391-3e04-4c58-841d-d933d520acc1 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:25.407843 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f08391-3e04-4c58-841d-d933d520acc1 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:25.407843 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f08391-3e04-4c58-841d-d933d520acc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:25.447953 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-99f08391-3e04-4c58-841d-d933d520acc1 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:25.449261 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-99f08391-3e04-4c58-841d-d933d520acc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:25.465761 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0fcb84-faf8-4dcc-ae69-79a3bd572777 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:25.469860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1030/2029] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:25.496176 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0fcb84-faf8-4dcc-ae69-79a3bd572777 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:25.506697 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d21081-e190-4201-ae00-ae7220fbbc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:25.507861 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d21081-e190-4201-ae00-ae7220fbbc6d admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:25.508362 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d21081-e190-4201-ae00-ae7220fbbc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:25.508940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d21081-e190-4201-ae00-ae7220fbbc6d admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:25.509889 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d21081-e190-4201-ae00-ae7220fbbc6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:34:25.510418 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d21081-e190-4201-ae00-ae7220fbbc6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:25.512859 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d21081-e190-4201-ae00-ae7220fbbc6d admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:25.515036 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:25.528735 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1000/2030] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:34:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:34:25.534024 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f08391-3e04-4c58-841d-d933d520acc1 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:25.547246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f08391-3e04-4c58-841d-d933d520acc1 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:25.553178 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9fe142-c13d-4bee-8ae3-6a78442b7ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:25.554164 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9fe142-c13d-4bee-8ae3-6a78442b7ecf admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:25.554614 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9fe142-c13d-4bee-8ae3-6a78442b7ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:25.555270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9fe142-c13d-4bee-8ae3-6a78442b7ecf admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:25.556357 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9fe142-c13d-4bee-8ae3-6a78442b7ecf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:25.558911 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9fe142-c13d-4bee-8ae3-6a78442b7ecf admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:25.695132 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1031/2031] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:34:25 2022] POST /identity/v3/projects => generated 377 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:34:25.696296 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d21081-e190-4201-ae00-ae7220fbbc6d admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:25.706506 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d21081-e190-4201-ae00-ae7220fbbc6d admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:25.766567 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9fe142-c13d-4bee-8ae3-6a78442b7ecf admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:25.769710 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-4a9fe142-c13d-4bee-8ae3-6a78442b7ecf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:25.770267 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a49ad20-0506-4b22-a6df-28d8ce6d3b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:25.771147 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49ad20-0506-4b22-a6df-28d8ce6d3b88 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:25.771812 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49ad20-0506-4b22-a6df-28d8ce6d3b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:25.772220 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49ad20-0506-4b22-a6df-28d8ce6d3b88 admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:25.773264 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a49ad20-0506-4b22-a6df-28d8ce6d3b88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:25.775642 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a49ad20-0506-4b22-a6df-28d8ce6d3b88 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:25.784259 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9fe142-c13d-4bee-8ae3-6a78442b7ecf admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:25.789334 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa6cffd-5dfd-4b1c-8933-f3474cd923b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:25.790252 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6cffd-5dfd-4b1c-8933-f3474cd923b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:25.790484 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6cffd-5dfd-4b1c-8933-f3474cd923b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:25.790854 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6cffd-5dfd-4b1c-8933-f3474cd923b4 admin admin] PATH_INFO: `/v3/users/332b1c9c68db4175a1383bb588b5e822` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:25.792117 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa6cffd-5dfd-4b1c-8933-f3474cd923b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=332b1c9c68db4175a1383bb588b5e822)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:25.794270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa6cffd-5dfd-4b1c-8933-f3474cd923b4 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:25.802821 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:25.918873 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1001/2032] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:34:25 2022] POST /identity/v3/users => generated 289 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:34:25.924780 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a49ad20-0506-4b22-a6df-28d8ce6d3b88 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:25.938750 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a49ad20-0506-4b22-a6df-28d8ce6d3b88 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:25.945507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e380941a-ad51-4d24-bb3b-6d5d6363e5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:25.946401 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e380941a-ad51-4d24-bb3b-6d5d6363e5f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:25.946688 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e380941a-ad51-4d24-bb3b-6d5d6363e5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:25.946965 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e380941a-ad51-4d24-bb3b-6d5d6363e5f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:25.947745 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e380941a-ad51-4d24-bb3b-6d5d6363e5f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:25.949349 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e380941a-ad51-4d24-bb3b-6d5d6363e5f7 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:25.960086 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1002/2033] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:34:25.964650 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e380941a-ad51-4d24-bb3b-6d5d6363e5f7 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:25.974895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e380941a-ad51-4d24-bb3b-6d5d6363e5f7 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:25.979590 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff76ade-e165-4c92-b0e9-f027427f938c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:25.980548 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff76ade-e165-4c92-b0e9-f027427f938c admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:25.980831 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff76ade-e165-4c92-b0e9-f027427f938c admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:25.981104 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff76ade-e165-4c92-b0e9-f027427f938c admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:25.981822 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff76ade-e165-4c92-b0e9-f027427f938c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:25.983383 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff76ade-e165-4c92-b0e9-f027427f938c admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:25.993749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1003/2034] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:34:25.997671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff76ade-e165-4c92-b0e9-f027427f938c admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:26.008004 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff76ade-e165-4c92-b0e9-f027427f938c admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:26.012048 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d1b66a-f2e9-4b75-95c7-fc8560eb435d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:26.012935 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1b66a-f2e9-4b75-95c7-fc8560eb435d admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:26.013213 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1b66a-f2e9-4b75-95c7-fc8560eb435d admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:26.013482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1b66a-f2e9-4b75-95c7-fc8560eb435d admin admin] PATH_INFO: `/v3/projects/c9a14dd5e74d4c7ca995940c31c223c8/users/90996c04d8e14924956fb172e108da58/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:26.019475 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-9aa6cffd-5dfd-4b1c-8933-f3474cd923b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '332b1c9c68db4175a1383bb588b5e822'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:26.020221 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-9aa6cffd-5dfd-4b1c-8933-f3474cd923b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '332b1c9c68db4175a1383bb588b5e822'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:26.027972 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-9aa6cffd-5dfd-4b1c-8933-f3474cd923b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '332b1c9c68db4175a1383bb588b5e822'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:26.029494 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-9aa6cffd-5dfd-4b1c-8933-f3474cd923b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '332b1c9c68db4175a1383bb588b5e822'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:26.033652 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d1b66a-f2e9-4b75-95c7-fc8560eb435d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9a14dd5e74d4c7ca995940c31c223c8, user_id=90996c04d8e14924956fb172e108da58, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:26.038940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d1b66a-f2e9-4b75-95c7-fc8560eb435d admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:26.055489 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-9aa6cffd-5dfd-4b1c-8933-f3474cd923b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '332b1c9c68db4175a1383bb588b5e822'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:26.109259 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1004/2035] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:34:25 2022] PUT /identity/v3/projects/c9a14dd5e74d4c7ca995940c31c223c8/users/90996c04d8e14924956fb172e108da58/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:34:26.120629 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fdd567-ad08-46b9-910a-181757c7da7e None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:26.120919 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fdd567-ad08-46b9-910a-181757c7da7e None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:26.121198 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fdd567-ad08-46b9-910a-181757c7da7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:26.148897 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-50fdd567-ad08-46b9-910a-181757c7da7e None None] MFA Rules not processed for user `90996c04d8e14924956fb172e108da58`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:26.150162 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-50fdd567-ad08-46b9-910a-181757c7da7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:26.168548 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1032/2036] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:34:25 2022] DELETE /identity/v3/users/332b1c9c68db4175a1383bb588b5e822 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:26.220869 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1005/2037] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:34:26.234999 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa6cffd-5dfd-4b1c-8933-f3474cd923b4 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:26.236668 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-9aa6cffd-5dfd-4b1c-8933-f3474cd923b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:26.302828 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa6cffd-5dfd-4b1c-8933-f3474cd923b4 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:26.306374 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d694446-c589-426a-bb3f-481b10769fd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:26.307067 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d694446-c589-426a-bb3f-481b10769fd2 service cinder] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:26.307351 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d694446-c589-426a-bb3f-481b10769fd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:26.307698 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d694446-c589-426a-bb3f-481b10769fd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:26.369400 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d694446-c589-426a-bb3f-481b10769fd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:26.370938 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d694446-c589-426a-bb3f-481b10769fd2 service cinder] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:26.381572 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1033/2038] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:34:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:34:26.464562 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fdd567-ad08-46b9-910a-181757c7da7e None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:26.466123 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-50fdd567-ad08-46b9-910a-181757c7da7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:26.547971 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:26.552019 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fdd567-ad08-46b9-910a-181757c7da7e None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:26.611139 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb4506-3165-49fe-a61a-57f56d5652e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:26.612044 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb4506-3165-49fe-a61a-57f56d5652e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:26.612516 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb4506-3165-49fe-a61a-57f56d5652e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:26.612779 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb4506-3165-49fe-a61a-57f56d5652e7 admin admin] PATH_INFO: `/v3/projects/b78a55fc7dba4ed5b22675b585cfe3aa` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:26.614056 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb4506-3165-49fe-a61a-57f56d5652e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b78a55fc7dba4ed5b22675b585cfe3aa)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:26.616418 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb4506-3165-49fe-a61a-57f56d5652e7 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:26.762073 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-accb4506-3165-49fe-a61a-57f56d5652e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b78a55fc7dba4ed5b22675b585cfe3aa', 'request_id': 'req-accb4506-3165-49fe-a61a-57f56d5652e7'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:26.767627 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-accb4506-3165-49fe-a61a-57f56d5652e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b78a55fc7dba4ed5b22675b585cfe3aa', 'request_id': 'req-accb4506-3165-49fe-a61a-57f56d5652e7'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:26.772090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d694446-c589-426a-bb3f-481b10769fd2 service cinder] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:26.773654 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-8d694446-c589-426a-bb3f-481b10769fd2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:26.777296 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-accb4506-3165-49fe-a61a-57f56d5652e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b78a55fc7dba4ed5b22675b585cfe3aa', 'request_id': 'req-accb4506-3165-49fe-a61a-57f56d5652e7'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:26.787427 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d694446-c589-426a-bb3f-481b10769fd2 service cinder] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:26.848101 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0975f90a-aa30-409b-95c9-ebbe2686113a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:26.848990 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0975f90a-aa30-409b-95c9-ebbe2686113a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:26.849275 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0975f90a-aa30-409b-95c9-ebbe2686113a admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:26.849554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0975f90a-aa30-409b-95c9-ebbe2686113a admin admin] PATH_INFO: `/v3/users/09ead7f054e344c7a965aec56e59dd86` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:26.850790 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0975f90a-aa30-409b-95c9-ebbe2686113a admin admin] RBAC: Authorizing `identity:delete_user(user_id=09ead7f054e344c7a965aec56e59dd86)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:26.852843 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0975f90a-aa30-409b-95c9-ebbe2686113a admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:26.862784 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-accb4506-3165-49fe-a61a-57f56d5652e7 admin admin] The token cache is being invalidate because project b78a55fc7dba4ed5b22675b585cfe3aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:34:26.863191 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-accb4506-3165-49fe-a61a-57f56d5652e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:26.863949 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-accb4506-3165-49fe-a61a-57f56d5652e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:26.866465 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1006/2039] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:34:26 2022] DELETE /identity/v3/projects/b78a55fc7dba4ed5b22675b585cfe3aa => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:26.871768 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d32d3-3a0b-4124-bc20-209914930a64 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:26.872480 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d32d3-3a0b-4124-bc20-209914930a64 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:26.872924 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d32d3-3a0b-4124-bc20-209914930a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:26.901467 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-a24d32d3-3a0b-4124-bc20-209914930a64 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:26.902705 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-a24d32d3-3a0b-4124-bc20-209914930a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:26.911367 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1007/2040] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:34:26.919880 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aaa822-f083-4187-b64a-20130d8ae92a None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:26.920184 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aaa822-f083-4187-b64a-20130d8ae92a None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:26.920463 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aaa822-f083-4187-b64a-20130d8ae92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:26.940315 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-c1aaa822-f083-4187-b64a-20130d8ae92a None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:26.941657 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-c1aaa822-f083-4187-b64a-20130d8ae92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:26.943231 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:26.949921 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1008/2041] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:34:26.953827 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aaa822-f083-4187-b64a-20130d8ae92a None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:26.962925 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aaa822-f083-4187-b64a-20130d8ae92a None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:26.968729 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3195df8-b58e-4885-9a65-feac4148572a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:26.969403 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3195df8-b58e-4885-9a65-feac4148572a admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:26.969985 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3195df8-b58e-4885-9a65-feac4148572a admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:26.970140 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3195df8-b58e-4885-9a65-feac4148572a admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:26.974617 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3195df8-b58e-4885-9a65-feac4148572a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:34:26.974617 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3195df8-b58e-4885-9a65-feac4148572a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:26.974617 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3195df8-b58e-4885-9a65-feac4148572a admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:26.986537 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1009/2042] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:34:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:34:26.990731 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3195df8-b58e-4885-9a65-feac4148572a admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:27.001773 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3195df8-b58e-4885-9a65-feac4148572a admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:27.006630 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e638c062-be7c-4bb8-a419-b52d004fd037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:27.007326 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638c062-be7c-4bb8-a419-b52d004fd037 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:27.007635 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638c062-be7c-4bb8-a419-b52d004fd037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:27.007911 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e638c062-be7c-4bb8-a419-b52d004fd037 admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:27.008714 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e638c062-be7c-4bb8-a419-b52d004fd037 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:27.010600 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e638c062-be7c-4bb8-a419-b52d004fd037 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:27.061616 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-0975f90a-aa30-409b-95c9-ebbe2686113a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09ead7f054e344c7a965aec56e59dd86'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:27.062561 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-0975f90a-aa30-409b-95c9-ebbe2686113a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09ead7f054e344c7a965aec56e59dd86'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:27.069873 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-0975f90a-aa30-409b-95c9-ebbe2686113a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09ead7f054e344c7a965aec56e59dd86'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:27.070544 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-0975f90a-aa30-409b-95c9-ebbe2686113a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09ead7f054e344c7a965aec56e59dd86'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:27.082390 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-0975f90a-aa30-409b-95c9-ebbe2686113a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09ead7f054e344c7a965aec56e59dd86'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:27.136210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1010/2043] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:34:26 2022] POST /identity/v3/projects => generated 389 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:34:27.137411 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e638c062-be7c-4bb8-a419-b52d004fd037 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:27.139211 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-e638c062-be7c-4bb8-a419-b52d004fd037 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:27.183111 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1034/2044] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:34:26 2022] DELETE /identity/v3/users/09ead7f054e344c7a965aec56e59dd86 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:34:27.190630 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e34af-0e9e-4176-adcc-2a20cab1e935 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:27.190926 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e34af-0e9e-4176-adcc-2a20cab1e935 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:27.191207 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e34af-0e9e-4176-adcc-2a20cab1e935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:27.208814 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e638c062-be7c-4bb8-a419-b52d004fd037 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:27.209764 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-775e34af-0e9e-4176-adcc-2a20cab1e935 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:27.211043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-775e34af-0e9e-4176-adcc-2a20cab1e935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:27.235135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:27.282724 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1035/2045] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:34:27.285954 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03473d-b8d4-4e2d-b37d-9533e187faef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:27.286770 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03473d-b8d4-4e2d-b37d-9533e187faef admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:27.287003 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03473d-b8d4-4e2d-b37d-9533e187faef admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:27.287281 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03473d-b8d4-4e2d-b37d-9533e187faef admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:27.288157 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd03473d-b8d4-4e2d-b37d-9533e187faef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:27.290299 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd03473d-b8d4-4e2d-b37d-9533e187faef admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:27.294686 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e34af-0e9e-4176-adcc-2a20cab1e935 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:27.297488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-775e34af-0e9e-4176-adcc-2a20cab1e935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:27.359410 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e34af-0e9e-4176-adcc-2a20cab1e935 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:27.363270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683a0745-e4f6-4e90-9d51-64602d1bc8a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:27.363952 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a0745-e4f6-4e90-9d51-64602d1bc8a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:27.364232 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a0745-e4f6-4e90-9d51-64602d1bc8a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:27.364504 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a0745-e4f6-4e90-9d51-64602d1bc8a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:27.377122 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683a0745-e4f6-4e90-9d51-64602d1bc8a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:27.378831 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683a0745-e4f6-4e90-9d51-64602d1bc8a4 service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:27.388966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1036/2046] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:34:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:34:27.393960 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1011/2047] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:34:27 2022] POST /identity/v3/users => generated 295 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:34:27.394100 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683a0745-e4f6-4e90-9d51-64602d1bc8a4 service neutron] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:27.403189 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683a0745-e4f6-4e90-9d51-64602d1bc8a4 service neutron] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:27.410481 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f99c2-d3c2-4a18-8385-272cd1076780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:27.411761 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f99c2-d3c2-4a18-8385-272cd1076780 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:27.411761 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f99c2-d3c2-4a18-8385-272cd1076780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:27.411761 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f99c2-d3c2-4a18-8385-272cd1076780 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:27.412652 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f99c2-d3c2-4a18-8385-272cd1076780 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:27.414378 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f99c2-d3c2-4a18-8385-272cd1076780 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:27.425675 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1037/2048] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:34:27.429472 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03473d-b8d4-4e2d-b37d-9533e187faef admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:27.440503 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03473d-b8d4-4e2d-b37d-9533e187faef admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:27.445502 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff03fc-3a6e-4896-9e71-4670c2bed9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:27.446275 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff03fc-3a6e-4896-9e71-4670c2bed9f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:27.446812 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff03fc-3a6e-4896-9e71-4670c2bed9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:27.447285 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff03fc-3a6e-4896-9e71-4670c2bed9f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:27.448126 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff03fc-3a6e-4896-9e71-4670c2bed9f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:27.449786 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff03fc-3a6e-4896-9e71-4670c2bed9f4 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:27.462443 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1012/2049] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:34:27.466248 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f99c2-d3c2-4a18-8385-272cd1076780 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:27.477638 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f99c2-d3c2-4a18-8385-272cd1076780 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:27.482690 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a7b39-baaf-4722-b37a-bc834b753185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:27.483968 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a7b39-baaf-4722-b37a-bc834b753185 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:27.483968 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a7b39-baaf-4722-b37a-bc834b753185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:27.484192 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a7b39-baaf-4722-b37a-bc834b753185 admin admin] PATH_INFO: `/v3/projects/8404651e9fb4443a9cdebb3f79a4e6a8/users/9528a9a3306d49b0978d77d1ce1ca900/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:27.499548 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95a7b39-baaf-4722-b37a-bc834b753185 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8404651e9fb4443a9cdebb3f79a4e6a8, user_id=9528a9a3306d49b0978d77d1ce1ca900, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:27.502346 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95a7b39-baaf-4722-b37a-bc834b753185 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:27.631115 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1038/2050] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:34:27 2022] PUT /identity/v3/projects/8404651e9fb4443a9cdebb3f79a4e6a8/users/9528a9a3306d49b0978d77d1ce1ca900/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:27.636285 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c81eb-bcdb-4c53-9546-41cb69e4bb11 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:27.636741 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c81eb-bcdb-4c53-9546-41cb69e4bb11 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:27.636894 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c81eb-bcdb-4c53-9546-41cb69e4bb11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:27.656724 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-055c81eb-bcdb-4c53-9546-41cb69e4bb11 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:27.658789 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-055c81eb-bcdb-4c53-9546-41cb69e4bb11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:27.728525 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1013/2051] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:27.738972 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a7b39-baaf-4722-b37a-bc834b753185 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:27.797458 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a7b39-baaf-4722-b37a-bc834b753185 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:27.801518 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac9ae4-8c3f-4038-a7fe-a0a872e5d949 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:27.802223 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac9ae4-8c3f-4038-a7fe-a0a872e5d949 service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:27.802506 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac9ae4-8c3f-4038-a7fe-a0a872e5d949 service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:27.802794 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac9ae4-8c3f-4038-a7fe-a0a872e5d949 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:27.815960 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dac9ae4-8c3f-4038-a7fe-a0a872e5d949 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:27.817405 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dac9ae4-8c3f-4038-a7fe-a0a872e5d949 service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:27.826892 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1039/2052] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:34:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:34:27.867357 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c81eb-bcdb-4c53-9546-41cb69e4bb11 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:27.868969 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-055c81eb-bcdb-4c53-9546-41cb69e4bb11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:27.884185 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:27.885374 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c81eb-bcdb-4c53-9546-41cb69e4bb11 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:27.893364 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492cf363-6dfb-4b0e-ac34-67ba5b26198e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:27.894245 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492cf363-6dfb-4b0e-ac34-67ba5b26198e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:27.894542 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492cf363-6dfb-4b0e-ac34-67ba5b26198e admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:27.894808 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492cf363-6dfb-4b0e-ac34-67ba5b26198e admin admin] PATH_INFO: `/v3/projects/3c3b09a5c1fe422d8318bf6a1a26c8ee` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:27.896057 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492cf363-6dfb-4b0e-ac34-67ba5b26198e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c3b09a5c1fe422d8318bf6a1a26c8ee)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:27.897981 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492cf363-6dfb-4b0e-ac34-67ba5b26198e admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:28.006721 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:28.016327 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-492cf363-6dfb-4b0e-ac34-67ba5b26198e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c3b09a5c1fe422d8318bf6a1a26c8ee', 'request_id': 'req-492cf363-6dfb-4b0e-ac34-67ba5b26198e'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:28.023256 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-492cf363-6dfb-4b0e-ac34-67ba5b26198e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c3b09a5c1fe422d8318bf6a1a26c8ee', 'request_id': 'req-492cf363-6dfb-4b0e-ac34-67ba5b26198e'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:28.031022 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-492cf363-6dfb-4b0e-ac34-67ba5b26198e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c3b09a5c1fe422d8318bf6a1a26c8ee', 'request_id': 'req-492cf363-6dfb-4b0e-ac34-67ba5b26198e'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:28.116864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-492cf363-6dfb-4b0e-ac34-67ba5b26198e admin admin] The token cache is being invalidate because project 3c3b09a5c1fe422d8318bf6a1a26c8ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:34:28.117153 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-492cf363-6dfb-4b0e-ac34-67ba5b26198e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:28.117810 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-492cf363-6dfb-4b0e-ac34-67ba5b26198e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:28.119394 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1014/2053] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:34:27 2022] DELETE /identity/v3/projects/3c3b09a5c1fe422d8318bf6a1a26c8ee => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:28.124421 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fac91-c7f2-4c2e-bcb5-89c725f9e2bf None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:28.124873 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fac91-c7f2-4c2e-bcb5-89c725f9e2bf None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:28.124999 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fac91-c7f2-4c2e-bcb5-89c725f9e2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:28.145434 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-315fac91-c7f2-4c2e-bcb5-89c725f9e2bf None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:28.145434 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-315fac91-c7f2-4c2e-bcb5-89c725f9e2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:28.216951 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1040/2054] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:28.219392 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946e039-e30e-46b1-83cf-21af4f1320be None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:28.219728 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946e039-e30e-46b1-83cf-21af4f1320be None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:28.219997 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946e039-e30e-46b1-83cf-21af4f1320be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:28.253964 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-f946e039-e30e-46b1-83cf-21af4f1320be None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:28.255179 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-f946e039-e30e-46b1-83cf-21af4f1320be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:28.265587 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1015/2055] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:28.269932 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315fac91-c7f2-4c2e-bcb5-89c725f9e2bf None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:28.281067 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315fac91-c7f2-4c2e-bcb5-89c725f9e2bf None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:28.285963 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a32af8-852e-4970-abf0-3c7bac80c836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:28.286623 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a32af8-852e-4970-abf0-3c7bac80c836 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:28.286920 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a32af8-852e-4970-abf0-3c7bac80c836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:28.287198 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a32af8-852e-4970-abf0-3c7bac80c836 admin admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:28.287965 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a32af8-852e-4970-abf0-3c7bac80c836 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:34:28.288278 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a32af8-852e-4970-abf0-3c7bac80c836 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:28.289880 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a32af8-852e-4970-abf0-3c7bac80c836 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:28.302304 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1041/2056] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:34:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:34:28.306593 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f946e039-e30e-46b1-83cf-21af4f1320be None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:28.316477 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f946e039-e30e-46b1-83cf-21af4f1320be None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:28.321485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c62bc2d-2812-43c4-9077-c57cea2a1d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:28.322353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62bc2d-2812-43c4-9077-c57cea2a1d2f admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:28.322636 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62bc2d-2812-43c4-9077-c57cea2a1d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:28.322906 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62bc2d-2812-43c4-9077-c57cea2a1d2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:28.323622 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c62bc2d-2812-43c4-9077-c57cea2a1d2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:28.325511 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c62bc2d-2812-43c4-9077-c57cea2a1d2f admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:28.435480 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1016/2057] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:34:28 2022] POST /identity/v3/projects => generated 377 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:34:28.440048 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a32af8-852e-4970-abf0-3c7bac80c836 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:28.461847 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a32af8-852e-4970-abf0-3c7bac80c836 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:28.522080 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4ac6f7-b74e-4697-b5e9-bbcbc203381c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:28.522783 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ac6f7-b74e-4697-b5e9-bbcbc203381c admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:28.523077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ac6f7-b74e-4697-b5e9-bbcbc203381c admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:28.523387 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ac6f7-b74e-4697-b5e9-bbcbc203381c admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:28.524205 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4ac6f7-b74e-4697-b5e9-bbcbc203381c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:28.526361 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4ac6f7-b74e-4697-b5e9-bbcbc203381c admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:28.629452 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1042/2058] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:34:28 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:34:28.636052 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c62bc2d-2812-43c4-9077-c57cea2a1d2f admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:28.646886 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c62bc2d-2812-43c4-9077-c57cea2a1d2f admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:28.651968 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:28.657675 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2d2e3b-26ed-4d83-8b01-7b6e528ba2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:28.666454 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d2e3b-26ed-4d83-8b01-7b6e528ba2b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:28.666454 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d2e3b-26ed-4d83-8b01-7b6e528ba2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:28.666454 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d2e3b-26ed-4d83-8b01-7b6e528ba2b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:28.671448 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2d2e3b-26ed-4d83-8b01-7b6e528ba2b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:28.675838 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2d2e3b-26ed-4d83-8b01-7b6e528ba2b7 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:28.691555 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1017/2059] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:28 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:34:28.699212 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4ac6f7-b74e-4697-b5e9-bbcbc203381c admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:28.712222 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4ac6f7-b74e-4697-b5e9-bbcbc203381c admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:28.717461 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d08af74-ea07-4713-82c9-1cf20317ce87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:28.718283 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08af74-ea07-4713-82c9-1cf20317ce87 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:28.718516 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08af74-ea07-4713-82c9-1cf20317ce87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:28.718803 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08af74-ea07-4713-82c9-1cf20317ce87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:28.719533 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d08af74-ea07-4713-82c9-1cf20317ce87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:28.724793 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d08af74-ea07-4713-82c9-1cf20317ce87 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:28.732356 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1043/2060] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:34:28.736616 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2d2e3b-26ed-4d83-8b01-7b6e528ba2b7 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:28.748801 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2d2e3b-26ed-4d83-8b01-7b6e528ba2b7 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:28.754037 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c036428-6fa0-4b2b-acb5-524d98a4545c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:28.754998 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c036428-6fa0-4b2b-acb5-524d98a4545c admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:28.755286 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c036428-6fa0-4b2b-acb5-524d98a4545c admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:28.755582 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c036428-6fa0-4b2b-acb5-524d98a4545c admin admin] PATH_INFO: `/v3/projects/c89f801372fa4cf1820721eb021726a1/users/8238e0ec5eaa4872a60a2814d33d7c4f/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:28.770635 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c036428-6fa0-4b2b-acb5-524d98a4545c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c89f801372fa4cf1820721eb021726a1, user_id=8238e0ec5eaa4872a60a2814d33d7c4f, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:28.773324 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c036428-6fa0-4b2b-acb5-524d98a4545c admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:28.889507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1018/2061] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:34:28 2022] PUT /identity/v3/projects/c89f801372fa4cf1820721eb021726a1/users/8238e0ec5eaa4872a60a2814d33d7c4f/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:28.900523 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be76f6-f772-4e52-8901-27cfa857ff07 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:28.900905 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be76f6-f772-4e52-8901-27cfa857ff07 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:28.901158 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be76f6-f772-4e52-8901-27cfa857ff07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:28.928973 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-c4be76f6-f772-4e52-8901-27cfa857ff07 None None] MFA Rules not processed for user `8238e0ec5eaa4872a60a2814d33d7c4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:28.930235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-c4be76f6-f772-4e52-8901-27cfa857ff07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:28.998357 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1044/2062] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:29.003833 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c036428-6fa0-4b2b-acb5-524d98a4545c admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:29.068556 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c036428-6fa0-4b2b-acb5-524d98a4545c admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:29.073273 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf227e4-d8b8-4754-b605-c77895e680ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:29.073929 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf227e4-d8b8-4754-b605-c77895e680ed admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:29.074216 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf227e4-d8b8-4754-b605-c77895e680ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:29.074497 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf227e4-d8b8-4754-b605-c77895e680ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:29.075180 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf227e4-d8b8-4754-b605-c77895e680ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:29.077144 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf227e4-d8b8-4754-b605-c77895e680ed admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:29.187596 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1019/2063] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:34:29 2022] POST /identity/v3/projects => generated 377 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:34:29.191796 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be76f6-f772-4e52-8901-27cfa857ff07 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:29.206641 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be76f6-f772-4e52-8901-27cfa857ff07 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:29.273114 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e115c0e-fcd2-412f-91d9-3326469e3391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:29.273817 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e115c0e-fcd2-412f-91d9-3326469e3391 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:29.274702 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e115c0e-fcd2-412f-91d9-3326469e3391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:29.274992 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e115c0e-fcd2-412f-91d9-3326469e3391 admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:29.275757 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e115c0e-fcd2-412f-91d9-3326469e3391 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:29.278022 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e115c0e-fcd2-412f-91d9-3326469e3391 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:29.400910 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1045/2064] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:34:29 2022] POST /identity/v3/users => generated 289 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:34:29.405068 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf227e4-d8b8-4754-b605-c77895e680ed admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:29.416968 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf227e4-d8b8-4754-b605-c77895e680ed admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:29.422440 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c35b66-6220-4f48-aee9-4cd0b6bf7984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:29.423269 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c35b66-6220-4f48-aee9-4cd0b6bf7984 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:29.423558 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c35b66-6220-4f48-aee9-4cd0b6bf7984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:29.423961 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c35b66-6220-4f48-aee9-4cd0b6bf7984 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:29.424903 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c35b66-6220-4f48-aee9-4cd0b6bf7984 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:29.426543 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c35b66-6220-4f48-aee9-4cd0b6bf7984 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:29.441874 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1020/2065] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:34:29.446353 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e115c0e-fcd2-412f-91d9-3326469e3391 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:29.459046 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e115c0e-fcd2-412f-91d9-3326469e3391 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:29.466875 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9586209b-66e2-4742-8c51-9f014b8af82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:29.467743 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9586209b-66e2-4742-8c51-9f014b8af82a admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:29.468990 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9586209b-66e2-4742-8c51-9f014b8af82a admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:29.468990 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9586209b-66e2-4742-8c51-9f014b8af82a admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:29.470750 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9586209b-66e2-4742-8c51-9f014b8af82a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:29.471701 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9586209b-66e2-4742-8c51-9f014b8af82a admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:29.484485 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1046/2066] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:34:29.488443 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c35b66-6220-4f48-aee9-4cd0b6bf7984 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:29.499819 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c35b66-6220-4f48-aee9-4cd0b6bf7984 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:29.504908 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca09f980-8fef-4ac4-8bb2-b95b9674b17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:29.505903 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca09f980-8fef-4ac4-8bb2-b95b9674b17b admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:29.506261 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca09f980-8fef-4ac4-8bb2-b95b9674b17b admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:29.506610 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca09f980-8fef-4ac4-8bb2-b95b9674b17b admin admin] PATH_INFO: `/v3/projects/c68a663d762143e5b0b1249d4de5ba14/users/82f2aea98d8143688369065c325b6047/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:29.525655 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca09f980-8fef-4ac4-8bb2-b95b9674b17b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c68a663d762143e5b0b1249d4de5ba14, user_id=82f2aea98d8143688369065c325b6047, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:29.528777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca09f980-8fef-4ac4-8bb2-b95b9674b17b admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:29.663810 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1021/2067] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:34:29 2022] PUT /identity/v3/projects/c68a663d762143e5b0b1249d4de5ba14/users/82f2aea98d8143688369065c325b6047/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:34:29.675597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9b275-d3ad-484e-8a65-a9ebfb819890 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:29.675893 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9b275-d3ad-484e-8a65-a9ebfb819890 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:29.676179 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9b275-d3ad-484e-8a65-a9ebfb819890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:29.705407 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-63e9b275-d3ad-484e-8a65-a9ebfb819890 None None] MFA Rules not processed for user `82f2aea98d8143688369065c325b6047`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:29.706633 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-63e9b275-d3ad-484e-8a65-a9ebfb819890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:29.758717 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:29.792729 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1047/2068] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:29.797664 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca09f980-8fef-4ac4-8bb2-b95b9674b17b admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:29.818476 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:29.876303 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca09f980-8fef-4ac4-8bb2-b95b9674b17b admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:29.881446 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa3343c-a0a9-4192-ac1e-32f0284c74d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:29.882204 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3343c-a0a9-4192-ac1e-32f0284c74d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:29.882561 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3343c-a0a9-4192-ac1e-32f0284c74d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:29.882898 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3343c-a0a9-4192-ac1e-32f0284c74d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:29.883709 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa3343c-a0a9-4192-ac1e-32f0284c74d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:29.885742 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa3343c-a0a9-4192-ac1e-32f0284c74d8 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:30.010072 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1022/2069] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:34:29 2022] POST /identity/v3/projects => generated 373 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:34:30.014133 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e9b275-d3ad-484e-8a65-a9ebfb819890 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:30.025061 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e9b275-d3ad-484e-8a65-a9ebfb819890 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:30.144411 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5434d2-cdc7-4f9d-9967-196b29830b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:30.155974 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5434d2-cdc7-4f9d-9967-196b29830b67 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:30.155974 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5434d2-cdc7-4f9d-9967-196b29830b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:30.155974 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5434d2-cdc7-4f9d-9967-196b29830b67 admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:30.155974 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5434d2-cdc7-4f9d-9967-196b29830b67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:30.155974 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5434d2-cdc7-4f9d-9967-196b29830b67 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:30.253367 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1048/2070] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:34:30 2022] POST /identity/v3/users => generated 293 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:34:30.258513 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa3343c-a0a9-4192-ac1e-32f0284c74d8 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:30.281907 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa3343c-a0a9-4192-ac1e-32f0284c74d8 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:30.290235 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0112a7-255f-458e-a0e9-d60fe821e801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:30.291194 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0112a7-255f-458e-a0e9-d60fe821e801 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:30.291664 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0112a7-255f-458e-a0e9-d60fe821e801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:30.292086 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0112a7-255f-458e-a0e9-d60fe821e801 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:30.294044 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0112a7-255f-458e-a0e9-d60fe821e801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:30.296776 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0112a7-255f-458e-a0e9-d60fe821e801 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:30.321038 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1023/2071] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:34:30.322738 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5434d2-cdc7-4f9d-9967-196b29830b67 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:30.334307 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5434d2-cdc7-4f9d-9967-196b29830b67 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:30.338989 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c929f06e-6410-49db-9b75-4708756f27db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:30.342943 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929f06e-6410-49db-9b75-4708756f27db admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:30.343322 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929f06e-6410-49db-9b75-4708756f27db admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:30.345618 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929f06e-6410-49db-9b75-4708756f27db admin admin] PATH_INFO: `/v3/projects/7c148bd923134156a6b00ac5735f01d6/users/e740f1658701473ba3b7f1e324b760b6/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:30.373720 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c929f06e-6410-49db-9b75-4708756f27db admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c148bd923134156a6b00ac5735f01d6, user_id=e740f1658701473ba3b7f1e324b760b6, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:30.377424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c929f06e-6410-49db-9b75-4708756f27db admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:30.446930 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1049/2072] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:34:30 2022] PUT /identity/v3/projects/7c148bd923134156a6b00ac5735f01d6/users/e740f1658701473ba3b7f1e324b760b6/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:30.451110 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0112a7-255f-458e-a0e9-d60fe821e801 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:30.461464 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0112a7-255f-458e-a0e9-d60fe821e801 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:30.517690 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ad561-8c01-4324-9304-c0c78fe506cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:30.518293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ad561-8c01-4324-9304-c0c78fe506cd admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:30.518578 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ad561-8c01-4324-9304-c0c78fe506cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:30.518857 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ad561-8c01-4324-9304-c0c78fe506cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:30.519646 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ad561-8c01-4324-9304-c0c78fe506cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:30.521269 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ad561-8c01-4324-9304-c0c78fe506cd admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:30.531392 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1024/2073] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:34:30.535450 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c929f06e-6410-49db-9b75-4708756f27db admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:30.545793 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c929f06e-6410-49db-9b75-4708756f27db admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:30.551346 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c12ed-f2a0-48ed-8754-27ef64c01022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:30.552263 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c12ed-f2a0-48ed-8754-27ef64c01022 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:30.552695 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c12ed-f2a0-48ed-8754-27ef64c01022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:30.552903 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c12ed-f2a0-48ed-8754-27ef64c01022 admin admin] PATH_INFO: `/v3/projects/7c148bd923134156a6b00ac5735f01d6/users/e740f1658701473ba3b7f1e324b760b6/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:30.554965 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c12ed-f2a0-48ed-8754-27ef64c01022 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c148bd923134156a6b00ac5735f01d6, user_id=e740f1658701473ba3b7f1e324b760b6, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:30.557692 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c12ed-f2a0-48ed-8754-27ef64c01022 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:30.569488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1050/2074] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:34:30 2022] PUT /identity/v3/projects/7c148bd923134156a6b00ac5735f01d6/users/e740f1658701473ba3b7f1e324b760b6/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:30.580574 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb5444-9bd6-4efc-9ef2-f70f93734d8c None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:30.580574 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb5444-9bd6-4efc-9ef2-f70f93734d8c None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:30.580574 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb5444-9bd6-4efc-9ef2-f70f93734d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:30.614215 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-4feb5444-9bd6-4efc-9ef2-f70f93734d8c None None] MFA Rules not processed for user `e740f1658701473ba3b7f1e324b760b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:30.615753 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-4feb5444-9bd6-4efc-9ef2-f70f93734d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:30.722702 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1025/2075] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:30 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:30.737830 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c12ed-f2a0-48ed-8754-27ef64c01022 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:30.749146 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-ca7c12ed-f2a0-48ed-8754-27ef64c01022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:30.828601 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c12ed-f2a0-48ed-8754-27ef64c01022 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:30.840757 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133d392c-2fbd-4cf1-a152-34091cb37e47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:30.841866 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d392c-2fbd-4cf1-a152-34091cb37e47 service cinder] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:30.842050 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d392c-2fbd-4cf1-a152-34091cb37e47 service cinder] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:30.852578 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d392c-2fbd-4cf1-a152-34091cb37e47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:30.900741 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:30.920325 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133d392c-2fbd-4cf1-a152-34091cb37e47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:30.922730 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133d392c-2fbd-4cf1-a152-34091cb37e47 service cinder] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:30.936049 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1051/2076] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:34:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:34:33.110923 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb5444-9bd6-4efc-9ef2-f70f93734d8c None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:33.130181 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb5444-9bd6-4efc-9ef2-f70f93734d8c None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:33.139196 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ddb967-ecdf-4a7b-8de0-ec15931fc3cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:33.142374 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddb967-ecdf-4a7b-8de0-ec15931fc3cb service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:33.143334 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddb967-ecdf-4a7b-8de0-ec15931fc3cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:33.143761 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddb967-ecdf-4a7b-8de0-ec15931fc3cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:33.156499 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:33.224557 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ddb967-ecdf-4a7b-8de0-ec15931fc3cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:33.226180 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ddb967-ecdf-4a7b-8de0-ec15931fc3cb service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:33.237099 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1026/2077] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:34:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:34:33.740179 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad3907-78bc-4211-9c9b-f488012230c9 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:33.741001 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad3907-78bc-4211-9c9b-f488012230c9 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:33.741426 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad3907-78bc-4211-9c9b-f488012230c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:33.771889 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-41ad3907-78bc-4211-9c9b-f488012230c9 None None] MFA Rules not processed for user `9528a9a3306d49b0978d77d1ce1ca900`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:33.773147 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-41ad3907-78bc-4211-9c9b-f488012230c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:33.855807 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1052/2078] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:33.866822 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bddff9f-f4b5-4f22-b4fa-1105b4b47a85 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:33.867133 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bddff9f-f4b5-4f22-b4fa-1105b4b47a85 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:33.867415 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bddff9f-f4b5-4f22-b4fa-1105b4b47a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:33.886692 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-0bddff9f-f4b5-4f22-b4fa-1105b4b47a85 None None] MFA Rules not processed for user `9528a9a3306d49b0978d77d1ce1ca900`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:33.887954 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-0bddff9f-f4b5-4f22-b4fa-1105b4b47a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:33.895040 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1027/2079] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:33.908521 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad3907-78bc-4211-9c9b-f488012230c9 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:33.909956 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-41ad3907-78bc-4211-9c9b-f488012230c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:33.990770 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad3907-78bc-4211-9c9b-f488012230c9 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:33.997842 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af4dbc-0b0b-479b-b7c6-a2a75023f733 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:33.999515 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af4dbc-0b0b-479b-b7c6-a2a75023f733 service nova] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:34.000012 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af4dbc-0b0b-479b-b7c6-a2a75023f733 service nova] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:34.000294 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af4dbc-0b0b-479b-b7c6-a2a75023f733 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:34.009214 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7af4dbc-0b0b-479b-b7c6-a2a75023f733 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:34.010839 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7af4dbc-0b0b-479b-b7c6-a2a75023f733 service nova] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:34.020663 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1053/2080] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:34:33 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:34:34.756024 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf5f6b-3b6d-4a10-bbbf-a0a95eb6b518 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:34.756838 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf5f6b-3b6d-4a10-bbbf-a0a95eb6b518 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:34.757175 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf5f6b-3b6d-4a10-bbbf-a0a95eb6b518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:34.779281 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-8ecf5f6b-3b6d-4a10-bbbf-a0a95eb6b518 None None] MFA Rules not processed for user `9528a9a3306d49b0978d77d1ce1ca900`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:34.781144 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-8ecf5f6b-3b6d-4a10-bbbf-a0a95eb6b518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:34.789998 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1028/2081] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:35.803488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af4dbc-0b0b-479b-b7c6-a2a75023f733 service nova] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:35.813817 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af4dbc-0b0b-479b-b7c6-a2a75023f733 service nova] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:35.819233 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f65ad-434f-4bad-90cb-978d74206a0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:35.820136 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f65ad-434f-4bad-90cb-978d74206a0b service cinder] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:35.820598 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f65ad-434f-4bad-90cb-978d74206a0b service cinder] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:35.821076 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f65ad-434f-4bad-90cb-978d74206a0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:35.835589 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467f65ad-434f-4bad-90cb-978d74206a0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:35.837324 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467f65ad-434f-4bad-90cb-978d74206a0b service cinder] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:35.848353 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1054/2082] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:34:35 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:34:40.915363 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf5f6b-3b6d-4a10-bbbf-a0a95eb6b518 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:40.916827 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-8ecf5f6b-3b6d-4a10-bbbf-a0a95eb6b518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:41.003855 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf5f6b-3b6d-4a10-bbbf-a0a95eb6b518 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:41.008680 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a08a7-f0c1-412e-8607-a5b2548061a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:41.009585 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a08a7-f0c1-412e-8607-a5b2548061a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:41.009874 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a08a7-f0c1-412e-8607-a5b2548061a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:41.010157 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a08a7-f0c1-412e-8607-a5b2548061a7 admin admin] PATH_INFO: `/v3/users/8f5c0d17b2354654aa87c2b656709952` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:41.011417 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a08a7-f0c1-412e-8607-a5b2548061a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f5c0d17b2354654aa87c2b656709952)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:41.013461 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a08a7-f0c1-412e-8607-a5b2548061a7 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:41.348761 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-605a08a7-f0c1-412e-8607-a5b2548061a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f5c0d17b2354654aa87c2b656709952'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:41.360305 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-605a08a7-f0c1-412e-8607-a5b2548061a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f5c0d17b2354654aa87c2b656709952'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:41.380187 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-605a08a7-f0c1-412e-8607-a5b2548061a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f5c0d17b2354654aa87c2b656709952'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:41.381126 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-605a08a7-f0c1-412e-8607-a5b2548061a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f5c0d17b2354654aa87c2b656709952'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:41.459803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-605a08a7-f0c1-412e-8607-a5b2548061a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f5c0d17b2354654aa87c2b656709952'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:41.469462 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1029/2083] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:34:40 2022] DELETE /identity/v3/users/8f5c0d17b2354654aa87c2b656709952 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:41.472850 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb146a65-ba7d-4a9d-a989-aa8e1d221c95 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:41.473311 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb146a65-ba7d-4a9d-a989-aa8e1d221c95 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:41.473397 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb146a65-ba7d-4a9d-a989-aa8e1d221c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:41.493596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-bb146a65-ba7d-4a9d-a989-aa8e1d221c95 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:41.494818 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-bb146a65-ba7d-4a9d-a989-aa8e1d221c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:41.535533 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:41.558686 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1055/2084] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:41.568733 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a08a7-f0c1-412e-8607-a5b2548061a7 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:41.571081 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-605a08a7-f0c1-412e-8607-a5b2548061a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:41.586748 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:41.635472 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a08a7-f0c1-412e-8607-a5b2548061a7 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:41.639709 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d6905-b40a-47ca-bc12-9dda7c8720e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:41.640364 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d6905-b40a-47ca-bc12-9dda7c8720e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:41.640649 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d6905-b40a-47ca-bc12-9dda7c8720e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:41.640955 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d6905-b40a-47ca-bc12-9dda7c8720e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:41.650862 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d6905-b40a-47ca-bc12-9dda7c8720e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:41.652403 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d6905-b40a-47ca-bc12-9dda7c8720e6 service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:41.664096 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1030/2085] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:34:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:34:41.757022 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb146a65-ba7d-4a9d-a989-aa8e1d221c95 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:41.758582 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-bb146a65-ba7d-4a9d-a989-aa8e1d221c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:41.772405 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb146a65-ba7d-4a9d-a989-aa8e1d221c95 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:41.779200 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da2c08-b62b-43f7-b715-eb4ef4660a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:41.780679 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da2c08-b62b-43f7-b715-eb4ef4660a03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:41.781246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da2c08-b62b-43f7-b715-eb4ef4660a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:41.782114 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da2c08-b62b-43f7-b715-eb4ef4660a03 admin admin] PATH_INFO: `/v3/users/8238e0ec5eaa4872a60a2814d33d7c4f` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:41.782744 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da2c08-b62b-43f7-b715-eb4ef4660a03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8238e0ec5eaa4872a60a2814d33d7c4f)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:41.785070 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da2c08-b62b-43f7-b715-eb4ef4660a03 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:42.046649 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-e5da2c08-b62b-43f7-b715-eb4ef4660a03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8238e0ec5eaa4872a60a2814d33d7c4f'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:42.047640 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-e5da2c08-b62b-43f7-b715-eb4ef4660a03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8238e0ec5eaa4872a60a2814d33d7c4f'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:42.054929 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-e5da2c08-b62b-43f7-b715-eb4ef4660a03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8238e0ec5eaa4872a60a2814d33d7c4f'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:42.055879 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-e5da2c08-b62b-43f7-b715-eb4ef4660a03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8238e0ec5eaa4872a60a2814d33d7c4f'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:42.069370 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-e5da2c08-b62b-43f7-b715-eb4ef4660a03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8238e0ec5eaa4872a60a2814d33d7c4f'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:42.132743 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d6905-b40a-47ca-bc12-9dda7c8720e6 service neutron] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:42.134850 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-032d6905-b40a-47ca-bc12-9dda7c8720e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:42.157721 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d6905-b40a-47ca-bc12-9dda7c8720e6 service neutron] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:42.164214 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1056/2086] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:34:41 2022] DELETE /identity/v3/users/8238e0ec5eaa4872a60a2814d33d7c4f => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:42.169536 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810183d-629e-4fec-a2f2-e37a1dfa96e3 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:42.169739 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810183d-629e-4fec-a2f2-e37a1dfa96e3 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:42.170113 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810183d-629e-4fec-a2f2-e37a1dfa96e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:42.172888 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecbb0bb-d71e-4299-9df4-8301ddf8af72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:42.173841 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecbb0bb-d71e-4299-9df4-8301ddf8af72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:42.180353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecbb0bb-d71e-4299-9df4-8301ddf8af72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:42.180353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecbb0bb-d71e-4299-9df4-8301ddf8af72 admin admin] PATH_INFO: `/v3/projects/04501c20d4f74c39873f862496f17c68` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:42.180353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecbb0bb-d71e-4299-9df4-8301ddf8af72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04501c20d4f74c39873f862496f17c68)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:42.180353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecbb0bb-d71e-4299-9df4-8301ddf8af72 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:42.190065 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-2810183d-629e-4fec-a2f2-e37a1dfa96e3 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:42.191293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-2810183d-629e-4fec-a2f2-e37a1dfa96e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:42.258788 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1057/2087] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:42.268795 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810183d-629e-4fec-a2f2-e37a1dfa96e3 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:42.270140 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-2810183d-629e-4fec-a2f2-e37a1dfa96e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:42.327522 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-3ecbb0bb-d71e-4299-9df4-8301ddf8af72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04501c20d4f74c39873f862496f17c68', 'request_id': 'req-3ecbb0bb-d71e-4299-9df4-8301ddf8af72'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:42.327766 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810183d-629e-4fec-a2f2-e37a1dfa96e3 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:42.333360 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-3ecbb0bb-d71e-4299-9df4-8301ddf8af72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04501c20d4f74c39873f862496f17c68', 'request_id': 'req-3ecbb0bb-d71e-4299-9df4-8301ddf8af72'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:42.336544 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:42.339777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-3ecbb0bb-d71e-4299-9df4-8301ddf8af72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04501c20d4f74c39873f862496f17c68', 'request_id': 'req-3ecbb0bb-d71e-4299-9df4-8301ddf8af72'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:42.379912 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa02f7-6c7a-495e-8f7b-62c18c473850 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:42.388662 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa02f7-6c7a-495e-8f7b-62c18c473850 service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:42.388662 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa02f7-6c7a-495e-8f7b-62c18c473850 service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:42.388662 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa02f7-6c7a-495e-8f7b-62c18c473850 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:42.417827 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-3ecbb0bb-d71e-4299-9df4-8301ddf8af72 admin admin] The token cache is being invalidate because project 04501c20d4f74c39873f862496f17c68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:34:42.418141 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-3ecbb0bb-d71e-4299-9df4-8301ddf8af72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:42.418845 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-3ecbb0bb-d71e-4299-9df4-8301ddf8af72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:42.421725 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1031/2088] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:34:42 2022] DELETE /identity/v3/projects/04501c20d4f74c39873f862496f17c68 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:42.427358 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ddc18-e4b2-46b8-b9d0-358114faddd8 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:42.427784 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ddc18-e4b2-46b8-b9d0-358114faddd8 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:42.427933 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ddc18-e4b2-46b8-b9d0-358114faddd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:42.453615 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-7f7ddc18-e4b2-46b8-b9d0-358114faddd8 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:42.454829 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-7f7ddc18-e4b2-46b8-b9d0-358114faddd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:42.456931 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aa02f7-6c7a-495e-8f7b-62c18c473850 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:42.458489 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aa02f7-6c7a-495e-8f7b-62c18c473850 service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:42.474349 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1032/2089] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:42.475911 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1058/2090] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:34:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:34:42.482176 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee117b-c921-45f0-b391-68ba8cd69a4d None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:42.482467 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee117b-c921-45f0-b391-68ba8cd69a4d None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:42.486775 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee117b-c921-45f0-b391-68ba8cd69a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:42.503767 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-73ee117b-c921-45f0-b391-68ba8cd69a4d None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:42.505249 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-73ee117b-c921-45f0-b391-68ba8cd69a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:42.522960 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1033/2091] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:42.524254 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa02f7-6c7a-495e-8f7b-62c18c473850 service neutron] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:42.525932 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-53aa02f7-6c7a-495e-8f7b-62c18c473850 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:42.542465 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa02f7-6c7a-495e-8f7b-62c18c473850 service neutron] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:42.556649 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b80109-8d4b-421d-bd8d-930521afaf74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:42.557539 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b80109-8d4b-421d-bd8d-930521afaf74 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:42.557862 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b80109-8d4b-421d-bd8d-930521afaf74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:42.558145 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b80109-8d4b-421d-bd8d-930521afaf74 admin admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:42.558911 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b80109-8d4b-421d-bd8d-930521afaf74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:34:42.559220 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b80109-8d4b-421d-bd8d-930521afaf74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:42.560877 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b80109-8d4b-421d-bd8d-930521afaf74 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:42.573585 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1059/2092] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:34:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:34:42.578601 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ee117b-c921-45f0-b391-68ba8cd69a4d None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:42.588084 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ee117b-c921-45f0-b391-68ba8cd69a4d None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:42.593312 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9111cd-28ee-41b0-a8cf-4af9c9939f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:42.593959 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9111cd-28ee-41b0-a8cf-4af9c9939f89 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:42.594241 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9111cd-28ee-41b0-a8cf-4af9c9939f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:42.594514 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9111cd-28ee-41b0-a8cf-4af9c9939f89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:42.595196 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9111cd-28ee-41b0-a8cf-4af9c9939f89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:42.597123 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9111cd-28ee-41b0-a8cf-4af9c9939f89 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:42.711206 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1034/2093] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:34:42 2022] POST /identity/v3/projects => generated 359 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:34:42.721284 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b80109-8d4b-421d-bd8d-930521afaf74 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:42.733745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b80109-8d4b-421d-bd8d-930521afaf74 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:42.806288 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c6088-17cc-47d1-b929-c0fdb2d16f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:42.807043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c6088-17cc-47d1-b929-c0fdb2d16f94 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:42.807397 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c6088-17cc-47d1-b929-c0fdb2d16f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:42.807773 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c6088-17cc-47d1-b929-c0fdb2d16f94 admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:42.808575 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285c6088-17cc-47d1-b929-c0fdb2d16f94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:42.811347 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285c6088-17cc-47d1-b929-c0fdb2d16f94 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:42.905414 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1060/2094] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:34:42 2022] POST /identity/v3/users => generated 280 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:34:42.909335 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9111cd-28ee-41b0-a8cf-4af9c9939f89 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:42.919437 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9111cd-28ee-41b0-a8cf-4af9c9939f89 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:42.924306 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf5101-1492-4f11-b86f-ce5266d17d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:42.924954 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf5101-1492-4f11-b86f-ce5266d17d59 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:42.925233 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf5101-1492-4f11-b86f-ce5266d17d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:42.925507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf5101-1492-4f11-b86f-ce5266d17d59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:42.926347 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cf5101-1492-4f11-b86f-ce5266d17d59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:42.928006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cf5101-1492-4f11-b86f-ce5266d17d59 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:42.939266 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1035/2095] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:34:42.943475 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c6088-17cc-47d1-b929-c0fdb2d16f94 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:42.962862 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf5101-1492-4f11-b86f-ce5266d17d59 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:42.971535 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-b8cf5101-1492-4f11-b86f-ce5266d17d59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:42.983395 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c6088-17cc-47d1-b929-c0fdb2d16f94 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:42.999145 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6889586-969c-40fd-b04a-1836fa5f5b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:42.999870 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6889586-969c-40fd-b04a-1836fa5f5b98 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:43.000153 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6889586-969c-40fd-b04a-1836fa5f5b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:43.000427 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6889586-969c-40fd-b04a-1836fa5f5b98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:43.001173 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6889586-969c-40fd-b04a-1836fa5f5b98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:43.003428 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6889586-969c-40fd-b04a-1836fa5f5b98 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:43.005681 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf5101-1492-4f11-b86f-ce5266d17d59 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:43.012111 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10699214-08db-4342-b590-e7637eda8765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:43.013082 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10699214-08db-4342-b590-e7637eda8765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:43.013443 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10699214-08db-4342-b590-e7637eda8765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:43.013792 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10699214-08db-4342-b590-e7637eda8765 admin admin] PATH_INFO: `/v3/projects/c89f801372fa4cf1820721eb021726a1` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:43.015910 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10699214-08db-4342-b590-e7637eda8765 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c89f801372fa4cf1820721eb021726a1)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:43.018107 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10699214-08db-4342-b590-e7637eda8765 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:43.020536 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1061/2096] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:34:43.021687 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6889586-969c-40fd-b04a-1836fa5f5b98 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:43.033662 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6889586-969c-40fd-b04a-1836fa5f5b98 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:43.041068 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ded5f-74cb-43bf-824a-ce8b418f76a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:43.042190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ded5f-74cb-43bf-824a-ce8b418f76a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:43.042674 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ded5f-74cb-43bf-824a-ce8b418f76a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:43.043105 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ded5f-74cb-43bf-824a-ce8b418f76a3 admin admin] PATH_INFO: `/v3/projects/9f44095ae47f444986c850c353460a70/users/4c2004a57a824ca3986c3526bdc3eb54/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:43.069296 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ded5f-74cb-43bf-824a-ce8b418f76a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f44095ae47f444986c850c353460a70, user_id=4c2004a57a824ca3986c3526bdc3eb54, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:43.072476 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ded5f-74cb-43bf-824a-ce8b418f76a3 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:43.196670 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1062/2097] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:34:43 2022] PUT /identity/v3/projects/9f44095ae47f444986c850c353460a70/users/4c2004a57a824ca3986c3526bdc3eb54/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:34:43.205545 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:43.208044 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c5cef-fae8-4ba3-b02c-3e8bb136f799 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:43.208402 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c5cef-fae8-4ba3-b02c-3e8bb136f799 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:43.208499 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-10699214-08db-4342-b590-e7637eda8765 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c89f801372fa4cf1820721eb021726a1', 'request_id': 'req-10699214-08db-4342-b590-e7637eda8765'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:43.208818 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c5cef-fae8-4ba3-b02c-3e8bb136f799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:43.214393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-10699214-08db-4342-b590-e7637eda8765 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c89f801372fa4cf1820721eb021726a1', 'request_id': 'req-10699214-08db-4342-b590-e7637eda8765'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:43.220760 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-10699214-08db-4342-b590-e7637eda8765 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c89f801372fa4cf1820721eb021726a1', 'request_id': 'req-10699214-08db-4342-b590-e7637eda8765'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:43.236554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-4c6c5cef-fae8-4ba3-b02c-3e8bb136f799 None None] MFA Rules not processed for user `4c2004a57a824ca3986c3526bdc3eb54`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:43.237783 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-4c6c5cef-fae8-4ba3-b02c-3e8bb136f799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:43.301837 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1063/2098] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:43 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:43.331396 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-10699214-08db-4342-b590-e7637eda8765 admin admin] The token cache is being invalidate because project c89f801372fa4cf1820721eb021726a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:34:43.331396 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-10699214-08db-4342-b590-e7637eda8765 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:43.331396 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-10699214-08db-4342-b590-e7637eda8765 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:43.331396 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1036/2099] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:34:42 2022] DELETE /identity/v3/projects/c89f801372fa4cf1820721eb021726a1 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:43.333800 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c5cef-fae8-4ba3-b02c-3e8bb136f799 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:43.336522 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10699214-08db-4342-b590-e7637eda8765 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:43.337957 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-10699214-08db-4342-b590-e7637eda8765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:43.351496 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:43.401405 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10699214-08db-4342-b590-e7637eda8765 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:43.405607 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b2098-4296-47b8-b154-a9c22be6b707 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:43.406443 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c5cef-fae8-4ba3-b02c-3e8bb136f799 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:43.406565 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b2098-4296-47b8-b154-a9c22be6b707 service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:43.406565 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b2098-4296-47b8-b154-a9c22be6b707 service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:43.406998 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b2098-4296-47b8-b154-a9c22be6b707 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:43.408776 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-bd3b2098-4296-47b8-b154-a9c22be6b707 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:43.411988 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f4318-ad3b-4101-aa94-94335589159f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:43.412857 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f4318-ad3b-4101-aa94-94335589159f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:43.413149 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f4318-ad3b-4101-aa94-94335589159f admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:43.413471 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f4318-ad3b-4101-aa94-94335589159f admin admin] PATH_INFO: `/v3/users/82f2aea98d8143688369065c325b6047` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:43.414610 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f4318-ad3b-4101-aa94-94335589159f admin admin] RBAC: Authorizing `identity:delete_user(user_id=82f2aea98d8143688369065c325b6047)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:43.416661 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f4318-ad3b-4101-aa94-94335589159f admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:43.421386 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b2098-4296-47b8-b154-a9c22be6b707 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:43.422824 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b2098-4296-47b8-b154-a9c22be6b707 service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:43.432851 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1037/2100] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:34:43 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:34:43.447725 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b2098-4296-47b8-b154-a9c22be6b707 service cinder] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:43.449531 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-bd3b2098-4296-47b8-b154-a9c22be6b707 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:43.462867 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b2098-4296-47b8-b154-a9c22be6b707 service cinder] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:43.468265 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3739bb-fac6-43ac-89c4-2b2214a320c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:43.469130 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3739bb-fac6-43ac-89c4-2b2214a320c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:43.469416 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3739bb-fac6-43ac-89c4-2b2214a320c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:43.469695 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3739bb-fac6-43ac-89c4-2b2214a320c6 admin admin] PATH_INFO: `/v3/users/4c2004a57a824ca3986c3526bdc3eb54` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:43.470923 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3739bb-fac6-43ac-89c4-2b2214a320c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c2004a57a824ca3986c3526bdc3eb54)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:43.472904 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3739bb-fac6-43ac-89c4-2b2214a320c6 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:43.679528 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-a30f4318-ad3b-4101-aa94-94335589159f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82f2aea98d8143688369065c325b6047'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:43.680357 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-a30f4318-ad3b-4101-aa94-94335589159f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82f2aea98d8143688369065c325b6047'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:43.687754 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-a30f4318-ad3b-4101-aa94-94335589159f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82f2aea98d8143688369065c325b6047'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:43.688794 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-a30f4318-ad3b-4101-aa94-94335589159f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82f2aea98d8143688369065c325b6047'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:43.702524 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-a30f4318-ad3b-4101-aa94-94335589159f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82f2aea98d8143688369065c325b6047'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:43.740091 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-5c3739bb-fac6-43ac-89c4-2b2214a320c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c2004a57a824ca3986c3526bdc3eb54'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:43.746960 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-5c3739bb-fac6-43ac-89c4-2b2214a320c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c2004a57a824ca3986c3526bdc3eb54'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:43.759948 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-5c3739bb-fac6-43ac-89c4-2b2214a320c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c2004a57a824ca3986c3526bdc3eb54'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:43.760679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-5c3739bb-fac6-43ac-89c4-2b2214a320c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c2004a57a824ca3986c3526bdc3eb54'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:43.805026 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1064/2101] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:34:43 2022] DELETE /identity/v3/users/82f2aea98d8143688369065c325b6047 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:43.925179 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-5c3739bb-fac6-43ac-89c4-2b2214a320c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c2004a57a824ca3986c3526bdc3eb54'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:43.931838 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1038/2102] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:34:43 2022] DELETE /identity/v3/users/4c2004a57a824ca3986c3526bdc3eb54 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:34:43.936531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e648d-f280-4bd7-ac6a-fa51f95a8674 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:43.936819 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e648d-f280-4bd7-ac6a-fa51f95a8674 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:43.937092 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e648d-f280-4bd7-ac6a-fa51f95a8674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:43.955323 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-da6e648d-f280-4bd7-ac6a-fa51f95a8674 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:43.956547 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-da6e648d-f280-4bd7-ac6a-fa51f95a8674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:44.016020 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1065/2103] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:44.026129 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3739bb-fac6-43ac-89c4-2b2214a320c6 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:44.027581 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-5c3739bb-fac6-43ac-89c4-2b2214a320c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:44.090842 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3739bb-fac6-43ac-89c4-2b2214a320c6 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:44.091289 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:44.095186 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde476c-fb0b-465b-9c8b-898d223de1ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:44.096767 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde476c-fb0b-465b-9c8b-898d223de1ac service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:44.096767 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde476c-fb0b-465b-9c8b-898d223de1ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:44.096767 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde476c-fb0b-465b-9c8b-898d223de1ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:44.106230 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cde476c-fb0b-465b-9c8b-898d223de1ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:44.107724 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cde476c-fb0b-465b-9c8b-898d223de1ac service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:44.118083 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1039/2104] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:34:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:34:44.148258 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:44.321249 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e648d-f280-4bd7-ac6a-fa51f95a8674 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:44.322701 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-da6e648d-f280-4bd7-ac6a-fa51f95a8674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:44.335467 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e648d-f280-4bd7-ac6a-fa51f95a8674 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:44.340730 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea04e8d5-ec3c-4ffa-801d-629e8aea0309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:44.341634 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04e8d5-ec3c-4ffa-801d-629e8aea0309 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:44.342057 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04e8d5-ec3c-4ffa-801d-629e8aea0309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:44.342337 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04e8d5-ec3c-4ffa-801d-629e8aea0309 admin admin] PATH_INFO: `/v3/projects/c68a663d762143e5b0b1249d4de5ba14` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:44.343608 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea04e8d5-ec3c-4ffa-801d-629e8aea0309 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c68a663d762143e5b0b1249d4de5ba14)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:44.345605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea04e8d5-ec3c-4ffa-801d-629e8aea0309 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:44.485888 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-ea04e8d5-ec3c-4ffa-801d-629e8aea0309 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c68a663d762143e5b0b1249d4de5ba14', 'request_id': 'req-ea04e8d5-ec3c-4ffa-801d-629e8aea0309'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:44.491380 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-ea04e8d5-ec3c-4ffa-801d-629e8aea0309 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c68a663d762143e5b0b1249d4de5ba14', 'request_id': 'req-ea04e8d5-ec3c-4ffa-801d-629e8aea0309'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:44.497709 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-ea04e8d5-ec3c-4ffa-801d-629e8aea0309 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c68a663d762143e5b0b1249d4de5ba14', 'request_id': 'req-ea04e8d5-ec3c-4ffa-801d-629e8aea0309'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:44.583919 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde476c-fb0b-465b-9c8b-898d223de1ac service neutron] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:44.588062 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-4cde476c-fb0b-465b-9c8b-898d223de1ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:44.597576 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-ea04e8d5-ec3c-4ffa-801d-629e8aea0309 admin admin] The token cache is being invalidate because project c68a663d762143e5b0b1249d4de5ba14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:34:44.597965 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-ea04e8d5-ec3c-4ffa-801d-629e8aea0309 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:44.598709 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-ea04e8d5-ec3c-4ffa-801d-629e8aea0309 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:44.600835 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1066/2105] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:34:44 2022] DELETE /identity/v3/projects/c68a663d762143e5b0b1249d4de5ba14 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:44.604726 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea04e8d5-ec3c-4ffa-801d-629e8aea0309 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:44.605589 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde476c-fb0b-465b-9c8b-898d223de1ac service neutron] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:44.607408 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-ea04e8d5-ec3c-4ffa-801d-629e8aea0309 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:44.679633 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e17cbf-66f7-4f3e-929d-85c0a48703db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:44.680589 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e17cbf-66f7-4f3e-929d-85c0a48703db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:44.680786 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea04e8d5-ec3c-4ffa-801d-629e8aea0309 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:44.681262 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e17cbf-66f7-4f3e-929d-85c0a48703db admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:44.681262 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e17cbf-66f7-4f3e-929d-85c0a48703db admin admin] PATH_INFO: `/v3/projects/9f44095ae47f444986c850c353460a70` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:44.682317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e17cbf-66f7-4f3e-929d-85c0a48703db admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f44095ae47f444986c850c353460a70)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:44.684279 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e17cbf-66f7-4f3e-929d-85c0a48703db admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:44.685536 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec7cf9-d129-48c1-97e1-e8aed834d8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:44.686989 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec7cf9-d129-48c1-97e1-e8aed834d8e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:44.687271 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec7cf9-d129-48c1-97e1-e8aed834d8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:44.687549 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec7cf9-d129-48c1-97e1-e8aed834d8e8 admin admin] PATH_INFO: `/v3/users/e740f1658701473ba3b7f1e324b760b6` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:44.688748 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec7cf9-d129-48c1-97e1-e8aed834d8e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e740f1658701473ba3b7f1e324b760b6)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:44.690642 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec7cf9-d129-48c1-97e1-e8aed834d8e8 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:44.808712 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-96e17cbf-66f7-4f3e-929d-85c0a48703db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f44095ae47f444986c850c353460a70', 'request_id': 'req-96e17cbf-66f7-4f3e-929d-85c0a48703db'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:44.814926 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-96e17cbf-66f7-4f3e-929d-85c0a48703db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f44095ae47f444986c850c353460a70', 'request_id': 'req-96e17cbf-66f7-4f3e-929d-85c0a48703db'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:44.821379 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-96e17cbf-66f7-4f3e-929d-85c0a48703db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f44095ae47f444986c850c353460a70', 'request_id': 'req-96e17cbf-66f7-4f3e-929d-85c0a48703db'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:44.890581 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-96e17cbf-66f7-4f3e-929d-85c0a48703db admin admin] The token cache is being invalidate because project 9f44095ae47f444986c850c353460a70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:34:44.890952 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-96e17cbf-66f7-4f3e-929d-85c0a48703db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:44.891579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-96e17cbf-66f7-4f3e-929d-85c0a48703db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:44.893155 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1040/2106] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:34:44 2022] DELETE /identity/v3/projects/9f44095ae47f444986c850c353460a70 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:34:44.898997 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff8937-90b7-4378-b1de-2301e8215b45 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:44.899308 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff8937-90b7-4378-b1de-2301e8215b45 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:44.899613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff8937-90b7-4378-b1de-2301e8215b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:44.917999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-9eff8937-90b7-4378-b1de-2301e8215b45 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:44.919187 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-9eff8937-90b7-4378-b1de-2301e8215b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:44.919706 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:44.928630 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-06ec7cf9-d129-48c1-97e1-e8aed834d8e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e740f1658701473ba3b7f1e324b760b6'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:44.929287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-06ec7cf9-d129-48c1-97e1-e8aed834d8e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e740f1658701473ba3b7f1e324b760b6'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:44.936492 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-06ec7cf9-d129-48c1-97e1-e8aed834d8e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e740f1658701473ba3b7f1e324b760b6'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:44.937276 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-06ec7cf9-d129-48c1-97e1-e8aed834d8e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e740f1658701473ba3b7f1e324b760b6'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:44.949288 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-06ec7cf9-d129-48c1-97e1-e8aed834d8e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e740f1658701473ba3b7f1e324b760b6'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:44.981622 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1041/2107] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:34:44.989628 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5fa0d-53e3-42ea-81db-dc252c460d6c None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:44.989916 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5fa0d-53e3-42ea-81db-dc252c460d6c None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:44.990190 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5fa0d-53e3-42ea-81db-dc252c460d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:45.007731 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-55a5fa0d-53e3-42ea-81db-dc252c460d6c None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:45.008934 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-55a5fa0d-53e3-42ea-81db-dc252c460d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:45.017038 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1042/2108] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:34:45.020822 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a5fa0d-53e3-42ea-81db-dc252c460d6c None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:45.030078 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a5fa0d-53e3-42ea-81db-dc252c460d6c None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:45.035913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32258de1-b7d9-4dcd-abd9-71a5415a0d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:45.036889 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32258de1-b7d9-4dcd-abd9-71a5415a0d99 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:45.037361 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32258de1-b7d9-4dcd-abd9-71a5415a0d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:45.037815 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32258de1-b7d9-4dcd-abd9-71a5415a0d99 admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:45.039042 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32258de1-b7d9-4dcd-abd9-71a5415a0d99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:34:45.039540 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32258de1-b7d9-4dcd-abd9-71a5415a0d99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:45.042100 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1067/2109] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:34:44 2022] DELETE /identity/v3/users/e740f1658701473ba3b7f1e324b760b6 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:34:45.042406 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32258de1-b7d9-4dcd-abd9-71a5415a0d99 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:45.061084 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1043/2110] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:34:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:34:45.061987 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec7cf9-d129-48c1-97e1-e8aed834d8e8 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:45.127779 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec7cf9-d129-48c1-97e1-e8aed834d8e8 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:45.132531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835f2f95-9022-431f-b616-481d497b30ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:45.133197 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f2f95-9022-431f-b616-481d497b30ff admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:45.133480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f2f95-9022-431f-b616-481d497b30ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:45.133759 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f2f95-9022-431f-b616-481d497b30ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:45.134459 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835f2f95-9022-431f-b616-481d497b30ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:45.136517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835f2f95-9022-431f-b616-481d497b30ff admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:45.252482 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1068/2111] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:34:45 2022] POST /identity/v3/projects => generated 365 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:34:45.256859 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32258de1-b7d9-4dcd-abd9-71a5415a0d99 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:45.262458 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:45.269864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32258de1-b7d9-4dcd-abd9-71a5415a0d99 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:45.329420 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a63405e-e273-406b-ac8a-e2a0967b787a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:45.330511 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63405e-e273-406b-ac8a-e2a0967b787a admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:45.330511 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63405e-e273-406b-ac8a-e2a0967b787a admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:45.330716 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63405e-e273-406b-ac8a-e2a0967b787a admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:45.331319 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a63405e-e273-406b-ac8a-e2a0967b787a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:45.333258 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a63405e-e273-406b-ac8a-e2a0967b787a admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:45.452571 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1044/2112] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:34:45 2022] POST /identity/v3/users => generated 283 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:34:45.456729 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835f2f95-9022-431f-b616-481d497b30ff admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:45.467620 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835f2f95-9022-431f-b616-481d497b30ff admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:45.472124 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a0c494-2492-4bc6-aadc-f345139e0b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:45.472892 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a0c494-2492-4bc6-aadc-f345139e0b57 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:45.473180 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a0c494-2492-4bc6-aadc-f345139e0b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:45.473672 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a0c494-2492-4bc6-aadc-f345139e0b57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:45.474503 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a0c494-2492-4bc6-aadc-f345139e0b57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:45.476140 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a0c494-2492-4bc6-aadc-f345139e0b57 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:45.485595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a63405e-e273-406b-ac8a-e2a0967b787a admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:45.487317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-9a63405e-e273-406b-ac8a-e2a0967b787a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:45.491176 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1069/2113] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:34:45.495455 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a0c494-2492-4bc6-aadc-f345139e0b57 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:45.506152 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a63405e-e273-406b-ac8a-e2a0967b787a admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:45.507134 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a0c494-2492-4bc6-aadc-f345139e0b57 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:45.512026 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b87038-d94e-4304-b9be-d514b6baa659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:45.512313 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b421b-7c74-4995-9e3f-68a5f6998361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:45.512936 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b87038-d94e-4304-b9be-d514b6baa659 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:45.513047 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b421b-7c74-4995-9e3f-68a5f6998361 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:45.513294 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b87038-d94e-4304-b9be-d514b6baa659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:45.513432 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b421b-7c74-4995-9e3f-68a5f6998361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:45.513685 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b421b-7c74-4995-9e3f-68a5f6998361 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:45.514302 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b87038-d94e-4304-b9be-d514b6baa659 admin admin] PATH_INFO: `/v3/projects/7c148bd923134156a6b00ac5735f01d6` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:45.514937 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459b421b-7c74-4995-9e3f-68a5f6998361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:45.515588 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b87038-d94e-4304-b9be-d514b6baa659 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c148bd923134156a6b00ac5735f01d6)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:45.516578 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459b421b-7c74-4995-9e3f-68a5f6998361 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:45.517668 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b87038-d94e-4304-b9be-d514b6baa659 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:45.531339 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1070/2114] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:34:45.532456 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b421b-7c74-4995-9e3f-68a5f6998361 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:45.542317 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b421b-7c74-4995-9e3f-68a5f6998361 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:45.547081 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1320a6-9c19-4168-b91a-5985f76cdf73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:45.547983 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1320a6-9c19-4168-b91a-5985f76cdf73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:45.548259 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1320a6-9c19-4168-b91a-5985f76cdf73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:45.548531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1320a6-9c19-4168-b91a-5985f76cdf73 admin admin] PATH_INFO: `/v3/projects/7d6edec41e0049b5903c78d2a77620dc/users/9bb4a2d7b0584fcdaaf296b31ad91ab6/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:45.563180 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1320a6-9c19-4168-b91a-5985f76cdf73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d6edec41e0049b5903c78d2a77620dc, user_id=9bb4a2d7b0584fcdaaf296b31ad91ab6, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:45.565909 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1320a6-9c19-4168-b91a-5985f76cdf73 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:45.664249 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1071/2115] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:34:45 2022] PUT /identity/v3/projects/7d6edec41e0049b5903c78d2a77620dc/users/9bb4a2d7b0584fcdaaf296b31ad91ab6/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:45.674544 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-19b87038-d94e-4304-b9be-d514b6baa659 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c148bd923134156a6b00ac5735f01d6', 'request_id': 'req-19b87038-d94e-4304-b9be-d514b6baa659'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:45.679357 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03aef4-f58b-4c96-baa6-a296b151259a None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:45.679783 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03aef4-f58b-4c96-baa6-a296b151259a None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:45.680187 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03aef4-f58b-4c96-baa6-a296b151259a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:45.681368 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-19b87038-d94e-4304-b9be-d514b6baa659 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c148bd923134156a6b00ac5735f01d6', 'request_id': 'req-19b87038-d94e-4304-b9be-d514b6baa659'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:45.690469 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-19b87038-d94e-4304-b9be-d514b6baa659 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c148bd923134156a6b00ac5735f01d6', 'request_id': 'req-19b87038-d94e-4304-b9be-d514b6baa659'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:45.711389 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-de03aef4-f58b-4c96-baa6-a296b151259a None None] MFA Rules not processed for user `9bb4a2d7b0584fcdaaf296b31ad91ab6`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:45.712639 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-de03aef4-f58b-4c96-baa6-a296b151259a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:45.760411 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:45.784158 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-19b87038-d94e-4304-b9be-d514b6baa659 admin admin] The token cache is being invalidate because project 7c148bd923134156a6b00ac5735f01d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:34:45.784387 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1072/2116] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:45 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:34:45.784483 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-19b87038-d94e-4304-b9be-d514b6baa659 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:45.785230 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-19b87038-d94e-4304-b9be-d514b6baa659 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:34:45.786987 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1045/2117] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:34:45 2022] DELETE /identity/v3/projects/7c148bd923134156a6b00ac5735f01d6 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:34:45.792145 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9bacc-c372-40d2-b3e3-e1b7a21ce1c2 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:45.792567 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9bacc-c372-40d2-b3e3-e1b7a21ce1c2 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:45.793010 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9bacc-c372-40d2-b3e3-e1b7a21ce1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:45.797139 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b87038-d94e-4304-b9be-d514b6baa659 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:45.801315 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-19b87038-d94e-4304-b9be-d514b6baa659 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:45.819501 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-a5f9bacc-c372-40d2-b3e3-e1b7a21ce1c2 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:45.820924 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-a5f9bacc-c372-40d2-b3e3-e1b7a21ce1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:45.864455 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b87038-d94e-4304-b9be-d514b6baa659 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:45.868095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5bff6b-dbb0-491d-84d1-f1ecc433f6cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:45.868731 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5bff6b-dbb0-491d-84d1-f1ecc433f6cc service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:45.869012 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5bff6b-dbb0-491d-84d1-f1ecc433f6cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:45.869281 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5bff6b-dbb0-491d-84d1-f1ecc433f6cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:45.871028 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-9b5bff6b-dbb0-491d-84d1-f1ecc433f6cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:45.882063 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5bff6b-dbb0-491d-84d1-f1ecc433f6cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:45.883476 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5bff6b-dbb0-491d-84d1-f1ecc433f6cc service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:45.894094 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1073/2118] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:45.894694 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1046/2119] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:34:45 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:34:45.949657 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a70a1-eb7f-417e-8b3b-7b8024a90ba7 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:45.949949 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a70a1-eb7f-417e-8b3b-7b8024a90ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:45.950247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a70a1-eb7f-417e-8b3b-7b8024a90ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:45.975099 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-079a70a1-eb7f-417e-8b3b-7b8024a90ba7 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:45.976552 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-079a70a1-eb7f-417e-8b3b-7b8024a90ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:45.987824 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1074/2120] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:45.991788 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5bff6b-dbb0-491d-84d1-f1ecc433f6cc service cinder] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:46.003480 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5bff6b-dbb0-491d-84d1-f1ecc433f6cc service cinder] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:46.013600 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dfae81-c798-45d9-add9-dadd86af7907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:46.015128 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dfae81-c798-45d9-add9-dadd86af7907 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:46.015556 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dfae81-c798-45d9-add9-dadd86af7907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:46.016002 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dfae81-c798-45d9-add9-dadd86af7907 admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:46.016890 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dfae81-c798-45d9-add9-dadd86af7907 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:34:46.017840 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dfae81-c798-45d9-add9-dadd86af7907 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:46.018946 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dfae81-c798-45d9-add9-dadd86af7907 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:46.033997 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1047/2121] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:34:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:34:46.038555 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079a70a1-eb7f-417e-8b3b-7b8024a90ba7 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:46.050245 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079a70a1-eb7f-417e-8b3b-7b8024a90ba7 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:46.055265 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fac700-028b-4c81-b148-aa129b6ac5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:46.056010 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fac700-028b-4c81-b148-aa129b6ac5e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:46.056276 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fac700-028b-4c81-b148-aa129b6ac5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:46.056553 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fac700-028b-4c81-b148-aa129b6ac5e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:46.057273 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fac700-028b-4c81-b148-aa129b6ac5e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:46.059406 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fac700-028b-4c81-b148-aa129b6ac5e4 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:46.171450 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1075/2122] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:34:46 2022] POST /identity/v3/projects => generated 385 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:34:46.175647 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dfae81-c798-45d9-add9-dadd86af7907 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:46.182040 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:46.186442 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dfae81-c798-45d9-add9-dadd86af7907 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:46.242614 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fdc44e-4b96-4f38-a7ed-6febc746b50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:46.243270 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fdc44e-4b96-4f38-a7ed-6febc746b50d admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:46.243555 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fdc44e-4b96-4f38-a7ed-6febc746b50d admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:46.243954 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fdc44e-4b96-4f38-a7ed-6febc746b50d admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:46.244535 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fdc44e-4b96-4f38-a7ed-6febc746b50d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:46.246442 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fdc44e-4b96-4f38-a7ed-6febc746b50d admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:46.333874 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1048/2123] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:34:46 2022] POST /identity/v3/users => generated 293 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:34:46.337957 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fac700-028b-4c81-b148-aa129b6ac5e4 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:46.348309 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fac700-028b-4c81-b148-aa129b6ac5e4 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:46.353063 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce70601-c11a-4992-bd6e-523b7358c2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:46.353713 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce70601-c11a-4992-bd6e-523b7358c2e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:46.354076 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce70601-c11a-4992-bd6e-523b7358c2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:46.354356 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce70601-c11a-4992-bd6e-523b7358c2e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:46.355376 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce70601-c11a-4992-bd6e-523b7358c2e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:46.356997 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce70601-c11a-4992-bd6e-523b7358c2e1 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:46.369488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1076/2124] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:34:46.374810 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fdc44e-4b96-4f38-a7ed-6febc746b50d admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:46.384812 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fdc44e-4b96-4f38-a7ed-6febc746b50d admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:46.390714 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019395f-cff3-4a08-b6f6-58352f04a211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:46.391373 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019395f-cff3-4a08-b6f6-58352f04a211 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:46.391690 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019395f-cff3-4a08-b6f6-58352f04a211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:46.391966 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019395f-cff3-4a08-b6f6-58352f04a211 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:46.392689 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d019395f-cff3-4a08-b6f6-58352f04a211 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:46.394280 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d019395f-cff3-4a08-b6f6-58352f04a211 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:46.405618 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1049/2125] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:34:46.409871 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce70601-c11a-4992-bd6e-523b7358c2e1 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:46.420609 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce70601-c11a-4992-bd6e-523b7358c2e1 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:46.425576 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0926d7a7-0393-41dd-aa09-36446e003831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:46.426440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926d7a7-0393-41dd-aa09-36446e003831 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:46.426721 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926d7a7-0393-41dd-aa09-36446e003831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:46.426997 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926d7a7-0393-41dd-aa09-36446e003831 admin admin] PATH_INFO: `/v3/projects/dc0ba2681ada4eb8b85942041465afca/users/b8fd5adee7c74687b23cee22696adf3a/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:46.442711 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0926d7a7-0393-41dd-aa09-36446e003831 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc0ba2681ada4eb8b85942041465afca, user_id=b8fd5adee7c74687b23cee22696adf3a, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:46.445396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0926d7a7-0393-41dd-aa09-36446e003831 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:46.585593 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1077/2126] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:34:46 2022] PUT /identity/v3/projects/dc0ba2681ada4eb8b85942041465afca/users/b8fd5adee7c74687b23cee22696adf3a/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:46.596419 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca90cf-9466-43ca-bcc9-98e55246f591 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:46.596727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca90cf-9466-43ca-bcc9-98e55246f591 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:46.597005 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca90cf-9466-43ca-bcc9-98e55246f591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:46.628229 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-57ca90cf-9466-43ca-bcc9-98e55246f591 None None] MFA Rules not processed for user `b8fd5adee7c74687b23cee22696adf3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:46.629622 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-57ca90cf-9466-43ca-bcc9-98e55246f591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:46.704104 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1050/2127] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:46.708527 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0926d7a7-0393-41dd-aa09-36446e003831 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:46.784620 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0926d7a7-0393-41dd-aa09-36446e003831 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:46.789358 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bfd413-c0bf-4a23-9010-fb17d997ccb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:46.790017 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bfd413-c0bf-4a23-9010-fb17d997ccb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:46.790300 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bfd413-c0bf-4a23-9010-fb17d997ccb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:46.790586 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bfd413-c0bf-4a23-9010-fb17d997ccb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:46.791310 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bfd413-c0bf-4a23-9010-fb17d997ccb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:46.793234 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bfd413-c0bf-4a23-9010-fb17d997ccb7 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:46.939203 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1078/2128] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:34:46 2022] POST /identity/v3/projects => generated 385 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:34:46.944752 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca90cf-9466-43ca-bcc9-98e55246f591 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:46.953531 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca90cf-9466-43ca-bcc9-98e55246f591 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:47.009786 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0be29-9b63-42d0-9a82-03903654c7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:47.010458 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0be29-9b63-42d0-9a82-03903654c7d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:47.010772 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0be29-9b63-42d0-9a82-03903654c7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:47.011053 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0be29-9b63-42d0-9a82-03903654c7d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:47.011776 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e0be29-9b63-42d0-9a82-03903654c7d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:47.013844 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e0be29-9b63-42d0-9a82-03903654c7d8 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:47.114720 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1051/2129] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:34:46 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:34:47.118524 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bfd413-c0bf-4a23-9010-fb17d997ccb7 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:47.126594 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:47.130533 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bfd413-c0bf-4a23-9010-fb17d997ccb7 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:47.136104 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8411f518-6860-4d5a-937f-c26390807027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:47.147758 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8411f518-6860-4d5a-937f-c26390807027 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:47.147758 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8411f518-6860-4d5a-937f-c26390807027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:47.147758 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8411f518-6860-4d5a-937f-c26390807027 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:47.147758 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8411f518-6860-4d5a-937f-c26390807027 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:47.147758 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8411f518-6860-4d5a-937f-c26390807027 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:47.152265 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1079/2130] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:34:47.156589 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0be29-9b63-42d0-9a82-03903654c7d8 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:47.167308 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0be29-9b63-42d0-9a82-03903654c7d8 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:47.173179 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4159a94-5e5a-41cb-b3fa-389e230270bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:47.173940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4159a94-5e5a-41cb-b3fa-389e230270bc admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:47.174226 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4159a94-5e5a-41cb-b3fa-389e230270bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:47.176852 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4159a94-5e5a-41cb-b3fa-389e230270bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:47.176852 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4159a94-5e5a-41cb-b3fa-389e230270bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:47.176852 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4159a94-5e5a-41cb-b3fa-389e230270bc admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:47.188208 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1052/2131] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:34:47.192502 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8411f518-6860-4d5a-937f-c26390807027 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:47.202793 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8411f518-6860-4d5a-937f-c26390807027 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:47.207902 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22776888-9700-451c-8f90-ca4be01fa590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:47.208865 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22776888-9700-451c-8f90-ca4be01fa590 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:47.209229 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22776888-9700-451c-8f90-ca4be01fa590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:47.209570 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22776888-9700-451c-8f90-ca4be01fa590 admin admin] PATH_INFO: `/v3/projects/134860c025634d3f9c8896d1456e672b/users/624d23a0ea5846a1a0b2c581bff9059f/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:47.226012 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22776888-9700-451c-8f90-ca4be01fa590 admin admin] RBAC: Authorizing `identity:create_grant(project_id=134860c025634d3f9c8896d1456e672b, user_id=624d23a0ea5846a1a0b2c581bff9059f, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:47.228826 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22776888-9700-451c-8f90-ca4be01fa590 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:47.344933 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1080/2132] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:34:47 2022] PUT /identity/v3/projects/134860c025634d3f9c8896d1456e672b/users/624d23a0ea5846a1a0b2c581bff9059f/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:34:47.356463 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e0486-d645-4e1f-94f2-a3477ceca855 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:47.356751 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e0486-d645-4e1f-94f2-a3477ceca855 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:47.357030 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e0486-d645-4e1f-94f2-a3477ceca855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:47.385749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-416e0486-d645-4e1f-94f2-a3477ceca855 None None] MFA Rules not processed for user `624d23a0ea5846a1a0b2c581bff9059f`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:47.387355 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-416e0486-d645-4e1f-94f2-a3477ceca855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:47.456334 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1053/2133] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:34:47.460725 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22776888-9700-451c-8f90-ca4be01fa590 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:47.523491 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22776888-9700-451c-8f90-ca4be01fa590 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:47.528335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f6eb5d-954f-45d2-a4d1-0fc57563963d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:47.529000 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6eb5d-954f-45d2-a4d1-0fc57563963d admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:47.529308 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6eb5d-954f-45d2-a4d1-0fc57563963d admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:47.529589 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6eb5d-954f-45d2-a4d1-0fc57563963d admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:47.530271 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f6eb5d-954f-45d2-a4d1-0fc57563963d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:47.532264 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f6eb5d-954f-45d2-a4d1-0fc57563963d admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:47.711160 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1081/2134] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:34:47 2022] POST /identity/v3/projects => generated 385 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:34:47.712702 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e0486-d645-4e1f-94f2-a3477ceca855 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:47.723447 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e0486-d645-4e1f-94f2-a3477ceca855 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:47.779282 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf884180-01fb-4080-8ec8-0499606d7165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:47.779982 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf884180-01fb-4080-8ec8-0499606d7165 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:47.780265 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf884180-01fb-4080-8ec8-0499606d7165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:47.780549 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf884180-01fb-4080-8ec8-0499606d7165 admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:47.781238 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf884180-01fb-4080-8ec8-0499606d7165 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:47.783180 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf884180-01fb-4080-8ec8-0499606d7165 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:47.902209 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1054/2135] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:34:47 2022] POST /identity/v3/users => generated 299 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:34:47.903368 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f6eb5d-954f-45d2-a4d1-0fc57563963d admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:47.914765 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f6eb5d-954f-45d2-a4d1-0fc57563963d admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:47.919337 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885086ef-6be5-473c-8918-418d442ea347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:47.920710 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885086ef-6be5-473c-8918-418d442ea347 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:47.921010 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885086ef-6be5-473c-8918-418d442ea347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:47.921267 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885086ef-6be5-473c-8918-418d442ea347 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:47.921995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885086ef-6be5-473c-8918-418d442ea347 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:47.923666 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885086ef-6be5-473c-8918-418d442ea347 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:47.934661 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1082/2136] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:34:47.938943 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf884180-01fb-4080-8ec8-0499606d7165 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:47.949084 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf884180-01fb-4080-8ec8-0499606d7165 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:47.953890 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be9d756-4fa4-4326-9f2f-9f1204290c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:47.955027 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9d756-4fa4-4326-9f2f-9f1204290c66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:47.955512 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9d756-4fa4-4326-9f2f-9f1204290c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:47.956001 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9d756-4fa4-4326-9f2f-9f1204290c66 admin admin] PATH_INFO: `/v3/projects/7c3bd3b536ae4b509505e962f87c343c/users/1f6f3a9933954aedb068cdad4d0725af/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:47.972249 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be9d756-4fa4-4326-9f2f-9f1204290c66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c3bd3b536ae4b509505e962f87c343c, user_id=1f6f3a9933954aedb068cdad4d0725af, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:47.974992 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be9d756-4fa4-4326-9f2f-9f1204290c66 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:47.985085 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:48.085327 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1055/2137] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:34:47 2022] PUT /identity/v3/projects/7c3bd3b536ae4b509505e962f87c343c/users/1f6f3a9933954aedb068cdad4d0725af/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:34:48.089424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885086ef-6be5-473c-8918-418d442ea347 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:48.100149 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885086ef-6be5-473c-8918-418d442ea347 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:48.154091 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b787dc-cf1b-48f0-ad79-5d049a0c3983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:48.155415 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b787dc-cf1b-48f0-ad79-5d049a0c3983 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:48.155753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b787dc-cf1b-48f0-ad79-5d049a0c3983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:48.156033 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b787dc-cf1b-48f0-ad79-5d049a0c3983 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:48.156756 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b787dc-cf1b-48f0-ad79-5d049a0c3983 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:48.158381 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b787dc-cf1b-48f0-ad79-5d049a0c3983 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:48.168246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1083/2138] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:34:48.173205 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be9d756-4fa4-4326-9f2f-9f1204290c66 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:48.189410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be9d756-4fa4-4326-9f2f-9f1204290c66 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:48.189410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643fe7f0-7b8f-4e6f-9048-ec47d2a94ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:48.189633 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643fe7f0-7b8f-4e6f-9048-ec47d2a94ca6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:48.189762 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643fe7f0-7b8f-4e6f-9048-ec47d2a94ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:48.190065 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643fe7f0-7b8f-4e6f-9048-ec47d2a94ca6 admin admin] PATH_INFO: `/v3/projects/7c3bd3b536ae4b509505e962f87c343c/users/1f6f3a9933954aedb068cdad4d0725af/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:48.192042 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643fe7f0-7b8f-4e6f-9048-ec47d2a94ca6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c3bd3b536ae4b509505e962f87c343c, user_id=1f6f3a9933954aedb068cdad4d0725af, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:48.194656 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643fe7f0-7b8f-4e6f-9048-ec47d2a94ca6 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:48.205305 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1056/2139] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:34:48 2022] PUT /identity/v3/projects/7c3bd3b536ae4b509505e962f87c343c/users/1f6f3a9933954aedb068cdad4d0725af/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:34:48.215430 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f0d8b-578b-4c7c-aae0-9711582a6fc1 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:48.215746 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f0d8b-578b-4c7c-aae0-9711582a6fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:48.216026 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f0d8b-578b-4c7c-aae0-9711582a6fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:48.241704 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-a44f0d8b-578b-4c7c-aae0-9711582a6fc1 None None] MFA Rules not processed for user `1f6f3a9933954aedb068cdad4d0725af`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:34:48.242899 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-a44f0d8b-578b-4c7c-aae0-9711582a6fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:48.309536 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1084/2140] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:34:48 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:34:48.322446 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643fe7f0-7b8f-4e6f-9048-ec47d2a94ca6 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:48.324402 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-643fe7f0-7b8f-4e6f-9048-ec47d2a94ca6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:34:48.368175 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:48.383775 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643fe7f0-7b8f-4e6f-9048-ec47d2a94ca6 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:48.387494 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d62599a-70a4-4153-b499-7580a0e544e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:48.388175 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62599a-70a4-4153-b499-7580a0e544e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:48.388464 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62599a-70a4-4153-b499-7580a0e544e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:48.388750 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62599a-70a4-4153-b499-7580a0e544e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:48.445606 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d62599a-70a4-4153-b499-7580a0e544e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:48.447030 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d62599a-70a4-4153-b499-7580a0e544e0 service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:48.455964 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1057/2141] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:34:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 15:34:48.933691 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:34:51.628405 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44f0d8b-578b-4c7c-aae0-9711582a6fc1 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:51.640086 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44f0d8b-578b-4c7c-aae0-9711582a6fc1 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:51.643812 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72dac66-ba38-4fe0-88fa-519df8146fcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:51.644454 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72dac66-ba38-4fe0-88fa-519df8146fcf service cinder] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:51.644737 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72dac66-ba38-4fe0-88fa-519df8146fcf service cinder] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:51.645033 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72dac66-ba38-4fe0-88fa-519df8146fcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:51.700915 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72dac66-ba38-4fe0-88fa-519df8146fcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:51.702973 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72dac66-ba38-4fe0-88fa-519df8146fcf service cinder] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:51.719650 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1085/2142] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:34:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:34:53.568731 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d62599a-70a4-4153-b499-7580a0e544e0 service cinder] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:34:53.578194 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d62599a-70a4-4153-b499-7580a0e544e0 service cinder] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:34:53.581864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c90e4-7154-4ce9-9683-b1466cf104ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:34:53.582552 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c90e4-7154-4ce9-9683-b1466cf104ed service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:34:53.582836 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c90e4-7154-4ce9-9683-b1466cf104ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:34:53.583111 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c90e4-7154-4ce9-9683-b1466cf104ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:34:53.596665 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768c90e4-7154-4ce9-9683-b1466cf104ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:34:53.598347 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768c90e4-7154-4ce9-9683-b1466cf104ed service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:34:53.609338 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1058/2143] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:34:53 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 15:35:00.597537 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72dac66-ba38-4fe0-88fa-519df8146fcf service cinder] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:00.597537 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-c72dac66-ba38-4fe0-88fa-519df8146fcf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:00.676693 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72dac66-ba38-4fe0-88fa-519df8146fcf service cinder] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:00.681953 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e8a0a9-25df-4b96-bb1d-6926e78e7867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:00.682955 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8a0a9-25df-4b96-bb1d-6926e78e7867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:00.683245 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8a0a9-25df-4b96-bb1d-6926e78e7867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:00.683519 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8a0a9-25df-4b96-bb1d-6926e78e7867 admin admin] PATH_INFO: `/v3/users/90996c04d8e14924956fb172e108da58` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:00.684733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e8a0a9-25df-4b96-bb1d-6926e78e7867 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90996c04d8e14924956fb172e108da58)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:00.687230 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e8a0a9-25df-4b96-bb1d-6926e78e7867 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:00.884378 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-68e8a0a9-25df-4b96-bb1d-6926e78e7867 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90996c04d8e14924956fb172e108da58'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:00.885025 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-68e8a0a9-25df-4b96-bb1d-6926e78e7867 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90996c04d8e14924956fb172e108da58'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:00.892716 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-68e8a0a9-25df-4b96-bb1d-6926e78e7867 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90996c04d8e14924956fb172e108da58'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:00.893356 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-68e8a0a9-25df-4b96-bb1d-6926e78e7867 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90996c04d8e14924956fb172e108da58'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:00.900601 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:35:00.906039 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-68e8a0a9-25df-4b96-bb1d-6926e78e7867 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90996c04d8e14924956fb172e108da58'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:01.033367 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1086/2144] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:35:00 2022] DELETE /identity/v3/users/90996c04d8e14924956fb172e108da58 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:35:01.037939 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2782b-65d2-4a98-af4e-ab0e5c440c5e None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:01.038410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2782b-65d2-4a98-af4e-ab0e5c440c5e None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:01.038507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2782b-65d2-4a98-af4e-ab0e5c440c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:01.056986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-6cb2782b-65d2-4a98-af4e-ab0e5c440c5e None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:35:01.058182 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-6cb2782b-65d2-4a98-af4e-ab0e5c440c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:01.121484 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1059/2145] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:35:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:35:01.133566 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e8a0a9-25df-4b96-bb1d-6926e78e7867 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:01.134996 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-68e8a0a9-25df-4b96-bb1d-6926e78e7867 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:01.195876 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e8a0a9-25df-4b96-bb1d-6926e78e7867 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:01.199486 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d58bcf-5d2d-4067-b7ec-d7e9a779f5ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:01.200216 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d58bcf-5d2d-4067-b7ec-d7e9a779f5ad service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:01.200489 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d58bcf-5d2d-4067-b7ec-d7e9a779f5ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:01.200757 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d58bcf-5d2d-4067-b7ec-d7e9a779f5ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:01.209881 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d58bcf-5d2d-4067-b7ec-d7e9a779f5ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:01.211320 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d58bcf-5d2d-4067-b7ec-d7e9a779f5ad service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:01.220688 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1087/2146] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:35:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:35:01.669717 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb2782b-65d2-4a98-af4e-ab0e5c440c5e None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:01.671999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-6cb2782b-65d2-4a98-af4e-ab0e5c440c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:01.688642 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb2782b-65d2-4a98-af4e-ab0e5c440c5e None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:01.694630 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939cc784-cd6a-4ff9-b9fb-563f4942f2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:01.696272 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939cc784-cd6a-4ff9-b9fb-563f4942f2fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:01.696565 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939cc784-cd6a-4ff9-b9fb-563f4942f2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:01.696846 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939cc784-cd6a-4ff9-b9fb-563f4942f2fa admin admin] PATH_INFO: `/v3/projects/c9a14dd5e74d4c7ca995940c31c223c8` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:01.698884 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939cc784-cd6a-4ff9-b9fb-563f4942f2fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9a14dd5e74d4c7ca995940c31c223c8)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:01.701364 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939cc784-cd6a-4ff9-b9fb-563f4942f2fa admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:01.737887 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d58bcf-5d2d-4067-b7ec-d7e9a779f5ad service neutron] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:01.739969 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-94d58bcf-5d2d-4067-b7ec-d7e9a779f5ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:01.758764 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d58bcf-5d2d-4067-b7ec-d7e9a779f5ad service neutron] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:01.763402 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83327408-fe88-4e7d-b122-192d00ce336b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:01.764835 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83327408-fe88-4e7d-b122-192d00ce336b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:01.765120 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83327408-fe88-4e7d-b122-192d00ce336b admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:01.765394 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83327408-fe88-4e7d-b122-192d00ce336b admin admin] PATH_INFO: `/v3/users/b8fd5adee7c74687b23cee22696adf3a` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:01.772819 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83327408-fe88-4e7d-b122-192d00ce336b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8fd5adee7c74687b23cee22696adf3a)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:01.772819 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83327408-fe88-4e7d-b122-192d00ce336b admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:01.830295 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-939cc784-cd6a-4ff9-b9fb-563f4942f2fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9a14dd5e74d4c7ca995940c31c223c8', 'request_id': 'req-939cc784-cd6a-4ff9-b9fb-563f4942f2fa'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:01.836622 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-939cc784-cd6a-4ff9-b9fb-563f4942f2fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9a14dd5e74d4c7ca995940c31c223c8', 'request_id': 'req-939cc784-cd6a-4ff9-b9fb-563f4942f2fa'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:01.844316 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-939cc784-cd6a-4ff9-b9fb-563f4942f2fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9a14dd5e74d4c7ca995940c31c223c8', 'request_id': 'req-939cc784-cd6a-4ff9-b9fb-563f4942f2fa'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:01.925703 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:35:01.930540 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-939cc784-cd6a-4ff9-b9fb-563f4942f2fa admin admin] The token cache is being invalidate because project c9a14dd5e74d4c7ca995940c31c223c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:35:01.930961 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-939cc784-cd6a-4ff9-b9fb-563f4942f2fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:01.931636 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-939cc784-cd6a-4ff9-b9fb-563f4942f2fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:01.933181 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1060/2147] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:35:01 2022] DELETE /identity/v3/projects/c9a14dd5e74d4c7ca995940c31c223c8 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:35:01.938081 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6efd5-7533-418e-8319-3dabc95a1ae9 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:01.938373 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6efd5-7533-418e-8319-3dabc95a1ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:01.938651 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6efd5-7533-418e-8319-3dabc95a1ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:01.957065 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-b0d6efd5-7533-418e-8319-3dabc95a1ae9 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:35:01.958262 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-b0d6efd5-7533-418e-8319-3dabc95a1ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:02.001190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-83327408-fe88-4e7d-b122-192d00ce336b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8fd5adee7c74687b23cee22696adf3a'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:02.001927 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-83327408-fe88-4e7d-b122-192d00ce336b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8fd5adee7c74687b23cee22696adf3a'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:02.009057 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-83327408-fe88-4e7d-b122-192d00ce336b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8fd5adee7c74687b23cee22696adf3a'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:02.009680 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-83327408-fe88-4e7d-b122-192d00ce336b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8fd5adee7c74687b23cee22696adf3a'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:02.020315 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1061/2148] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:35:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:35:02.028012 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c3b7a-921e-4795-bf22-210c8dd06e35 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:02.028302 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c3b7a-921e-4795-bf22-210c8dd06e35 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:02.028469 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-83327408-fe88-4e7d-b122-192d00ce336b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8fd5adee7c74687b23cee22696adf3a'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:02.028570 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c3b7a-921e-4795-bf22-210c8dd06e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:02.047124 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-796c3b7a-921e-4795-bf22-210c8dd06e35 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:35:02.053803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-796c3b7a-921e-4795-bf22-210c8dd06e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:02.056357 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1062/2149] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:35:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:35:02.060316 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c3b7a-921e-4795-bf22-210c8dd06e35 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:02.068878 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c3b7a-921e-4795-bf22-210c8dd06e35 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:02.073976 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395d746-9224-4395-90eb-58202f832392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:02.074595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395d746-9224-4395-90eb-58202f832392 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:02.074891 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395d746-9224-4395-90eb-58202f832392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:02.075161 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395d746-9224-4395-90eb-58202f832392 admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:02.075955 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5395d746-9224-4395-90eb-58202f832392 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:35:02.076780 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5395d746-9224-4395-90eb-58202f832392 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:02.082537 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5395d746-9224-4395-90eb-58202f832392 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:02.090236 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1063/2150] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:35:02.094861 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395d746-9224-4395-90eb-58202f832392 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:02.104282 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395d746-9224-4395-90eb-58202f832392 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:02.107307 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1088/2151] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:35:01 2022] DELETE /identity/v3/users/b8fd5adee7c74687b23cee22696adf3a => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:35:02.111498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626d67d-1f83-40ce-8183-5b70fac5034d None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:02.111812 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626d67d-1f83-40ce-8183-5b70fac5034d None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:02.112083 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626d67d-1f83-40ce-8183-5b70fac5034d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:02.112872 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39fd9e-81d6-4001-9db6-dd60dd75ac7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:02.113637 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39fd9e-81d6-4001-9db6-dd60dd75ac7b admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:02.116926 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39fd9e-81d6-4001-9db6-dd60dd75ac7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:02.116926 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39fd9e-81d6-4001-9db6-dd60dd75ac7b admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:02.116926 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39fd9e-81d6-4001-9db6-dd60dd75ac7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:02.116926 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39fd9e-81d6-4001-9db6-dd60dd75ac7b admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:02.129802 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-d626d67d-1f83-40ce-8183-5b70fac5034d None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:35:02.131014 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-d626d67d-1f83-40ce-8183-5b70fac5034d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:02.179956 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:35:02.189474 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1089/2152] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:35:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:35:02.199009 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d626d67d-1f83-40ce-8183-5b70fac5034d None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:02.200492 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-d626d67d-1f83-40ce-8183-5b70fac5034d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:02.204328 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1064/2153] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:35:02 2022] POST /identity/v3/projects => generated 369 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:35:02.208104 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39fd9e-81d6-4001-9db6-dd60dd75ac7b admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:02.261627 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d626d67d-1f83-40ce-8183-5b70fac5034d None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:02.271303 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39fd9e-81d6-4001-9db6-dd60dd75ac7b admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:02.276171 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34680d7-5a63-4978-9709-73dcdf4f6a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:02.276836 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34680d7-5a63-4978-9709-73dcdf4f6a31 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:02.277140 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34680d7-5a63-4978-9709-73dcdf4f6a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:02.277419 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34680d7-5a63-4978-9709-73dcdf4f6a31 admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:02.278104 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34680d7-5a63-4978-9709-73dcdf4f6a31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:02.280167 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34680d7-5a63-4978-9709-73dcdf4f6a31 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:02.309440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2328903-ac2b-451f-82c3-95cf74e702cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:02.310095 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2328903-ac2b-451f-82c3-95cf74e702cb service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:02.310356 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2328903-ac2b-451f-82c3-95cf74e702cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:02.311174 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2328903-ac2b-451f-82c3-95cf74e702cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:02.320049 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2328903-ac2b-451f-82c3-95cf74e702cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:02.321468 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2328903-ac2b-451f-82c3-95cf74e702cb service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:02.331338 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1090/2154] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:35:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 15:35:02.401007 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1065/2155] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:35:02 2022] POST /identity/v3/users => generated 285 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:35:02.405017 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2328903-ac2b-451f-82c3-95cf74e702cb service neutron] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:02.415415 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2328903-ac2b-451f-82c3-95cf74e702cb service neutron] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:02.421328 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d67b8-72ac-4bae-a341-5ed211657399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:02.422321 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d67b8-72ac-4bae-a341-5ed211657399 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:02.422321 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d67b8-72ac-4bae-a341-5ed211657399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:02.422690 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d67b8-72ac-4bae-a341-5ed211657399 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:02.423366 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78d67b8-72ac-4bae-a341-5ed211657399 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:02.425035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78d67b8-72ac-4bae-a341-5ed211657399 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:02.439904 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1091/2156] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:35:02.441143 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34680d7-5a63-4978-9709-73dcdf4f6a31 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:02.457055 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34680d7-5a63-4978-9709-73dcdf4f6a31 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:02.462815 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c676977-93eb-4b3f-819f-15ef92d0b439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:02.463722 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c676977-93eb-4b3f-819f-15ef92d0b439 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:02.463999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c676977-93eb-4b3f-819f-15ef92d0b439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:02.464284 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c676977-93eb-4b3f-819f-15ef92d0b439 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:02.465028 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c676977-93eb-4b3f-819f-15ef92d0b439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:02.466652 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c676977-93eb-4b3f-819f-15ef92d0b439 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:02.481265 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1066/2157] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:35:02.482262 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d67b8-72ac-4bae-a341-5ed211657399 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:02.492588 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d67b8-72ac-4bae-a341-5ed211657399 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:02.499112 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9727f3-012b-4cd2-bdbc-dcf94cb34813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:02.500039 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9727f3-012b-4cd2-bdbc-dcf94cb34813 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:02.500320 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9727f3-012b-4cd2-bdbc-dcf94cb34813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:02.500599 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9727f3-012b-4cd2-bdbc-dcf94cb34813 admin admin] PATH_INFO: `/v3/projects/4c9f51214c284f47a318c428797e4bcc/users/70590fe3a00a4d56bcb61f99f52c86e8/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:02.517293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9727f3-012b-4cd2-bdbc-dcf94cb34813 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c9f51214c284f47a318c428797e4bcc, user_id=70590fe3a00a4d56bcb61f99f52c86e8, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:02.520605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9727f3-012b-4cd2-bdbc-dcf94cb34813 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:02.602209 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1092/2158] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:35:02 2022] PUT /identity/v3/projects/4c9f51214c284f47a318c428797e4bcc/users/70590fe3a00a4d56bcb61f99f52c86e8/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:35:02.613709 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d4e3f-d6a5-47b5-ada6-33108122b29c None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:02.613999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d4e3f-d6a5-47b5-ada6-33108122b29c None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:02.614277 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d4e3f-d6a5-47b5-ada6-33108122b29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:02.642711 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-218d4e3f-d6a5-47b5-ada6-33108122b29c None None] MFA Rules not processed for user `70590fe3a00a4d56bcb61f99f52c86e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:35:02.643979 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-218d4e3f-d6a5-47b5-ada6-33108122b29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:02.668355 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:35:02.708099 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1067/2159] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:35:02 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:35:02.719757 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9727f3-012b-4cd2-bdbc-dcf94cb34813 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:02.721424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-ec9727f3-012b-4cd2-bdbc-dcf94cb34813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:02.737885 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218d4e3f-d6a5-47b5-ada6-33108122b29c None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:02.740071 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-218d4e3f-d6a5-47b5-ada6-33108122b29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:02.815945 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9727f3-012b-4cd2-bdbc-dcf94cb34813 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:02.820285 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d09ce-e80e-4ddd-9e75-c14869abc3c8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0d29e718f446e2a9fd1b2575a86041', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:02.820440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d09ce-e80e-4ddd-9e75-c14869abc3c8 service glance] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:02.820705 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d09ce-e80e-4ddd-9e75-c14869abc3c8 service glance] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:02.820987 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d09ce-e80e-4ddd-9e75-c14869abc3c8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:02.824070 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218d4e3f-d6a5-47b5-ada6-33108122b29c None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:02.828745 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dbf26f-9b5f-4825-827f-04a5fe790dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:02.829599 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dbf26f-9b5f-4825-827f-04a5fe790dec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:02.829887 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dbf26f-9b5f-4825-827f-04a5fe790dec admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:02.830164 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dbf26f-9b5f-4825-827f-04a5fe790dec admin admin] PATH_INFO: `/v3/projects/dc0ba2681ada4eb8b85942041465afca` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:02.831353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dbf26f-9b5f-4825-827f-04a5fe790dec admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc0ba2681ada4eb8b85942041465afca)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:02.831596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908d09ce-e80e-4ddd-9e75-c14869abc3c8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:02.833110 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908d09ce-e80e-4ddd-9e75-c14869abc3c8 service glance] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:02.833336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dbf26f-9b5f-4825-827f-04a5fe790dec admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:02.842700 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1093/2160] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:35:02 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:35:03.010005 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-71dbf26f-9b5f-4825-827f-04a5fe790dec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc0ba2681ada4eb8b85942041465afca', 'request_id': 'req-71dbf26f-9b5f-4825-827f-04a5fe790dec'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:03.022348 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-71dbf26f-9b5f-4825-827f-04a5fe790dec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc0ba2681ada4eb8b85942041465afca', 'request_id': 'req-71dbf26f-9b5f-4825-827f-04a5fe790dec'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:03.041373 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-71dbf26f-9b5f-4825-827f-04a5fe790dec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc0ba2681ada4eb8b85942041465afca', 'request_id': 'req-71dbf26f-9b5f-4825-827f-04a5fe790dec'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:03.113698 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-71dbf26f-9b5f-4825-827f-04a5fe790dec admin admin] The token cache is being invalidate because project dc0ba2681ada4eb8b85942041465afca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:35:03.114615 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-71dbf26f-9b5f-4825-827f-04a5fe790dec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:03.115457 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-71dbf26f-9b5f-4825-827f-04a5fe790dec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:03.117794 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1068/2161] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:35:02 2022] DELETE /identity/v3/projects/dc0ba2681ada4eb8b85942041465afca => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:35:03.121747 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d09ce-e80e-4ddd-9e75-c14869abc3c8 service glance] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:03.134742 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d09ce-e80e-4ddd-9e75-c14869abc3c8 service glance] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:03.157027 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:35:03.195231 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7fb88-1e65-43de-8df3-cd2c37d3d1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:03.196117 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7fb88-1e65-43de-8df3-cd2c37d3d1e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:03.196401 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7fb88-1e65-43de-8df3-cd2c37d3d1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:03.196677 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7fb88-1e65-43de-8df3-cd2c37d3d1e5 admin admin] PATH_INFO: `/v3/users/624d23a0ea5846a1a0b2c581bff9059f` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:03.197856 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c7fb88-1e65-43de-8df3-cd2c37d3d1e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=624d23a0ea5846a1a0b2c581bff9059f)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:03.199806 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c7fb88-1e65-43de-8df3-cd2c37d3d1e5 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:03.417538 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-01c7fb88-1e65-43de-8df3-cd2c37d3d1e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '624d23a0ea5846a1a0b2c581bff9059f'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:03.418192 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-01c7fb88-1e65-43de-8df3-cd2c37d3d1e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '624d23a0ea5846a1a0b2c581bff9059f'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:03.425560 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-01c7fb88-1e65-43de-8df3-cd2c37d3d1e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '624d23a0ea5846a1a0b2c581bff9059f'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:03.426300 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-01c7fb88-1e65-43de-8df3-cd2c37d3d1e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '624d23a0ea5846a1a0b2c581bff9059f'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:03.445684 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-01c7fb88-1e65-43de-8df3-cd2c37d3d1e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '624d23a0ea5846a1a0b2c581bff9059f'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:03.559176 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1094/2162] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:35:03 2022] DELETE /identity/v3/users/624d23a0ea5846a1a0b2c581bff9059f => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:35:03.980782 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dbf26f-9b5f-4825-827f-04a5fe790dec admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:03.982222 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-71dbf26f-9b5f-4825-827f-04a5fe790dec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:03.995383 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dbf26f-9b5f-4825-827f-04a5fe790dec admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:04.057860 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:35:04.066716 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0158d88-1ad6-4a2e-8fe6-66b13f0a350d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:04.066716 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0158d88-1ad6-4a2e-8fe6-66b13f0a350d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:04.066716 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0158d88-1ad6-4a2e-8fe6-66b13f0a350d admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:04.067051 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0158d88-1ad6-4a2e-8fe6-66b13f0a350d admin admin] PATH_INFO: `/v3/projects/134860c025634d3f9c8896d1456e672b` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:04.068130 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0158d88-1ad6-4a2e-8fe6-66b13f0a350d admin admin] RBAC: Authorizing `identity:delete_project(project_id=134860c025634d3f9c8896d1456e672b)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:04.070006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0158d88-1ad6-4a2e-8fe6-66b13f0a350d admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:04.212678 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-c0158d88-1ad6-4a2e-8fe6-66b13f0a350d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '134860c025634d3f9c8896d1456e672b', 'request_id': 'req-c0158d88-1ad6-4a2e-8fe6-66b13f0a350d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:04.221663 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-c0158d88-1ad6-4a2e-8fe6-66b13f0a350d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '134860c025634d3f9c8896d1456e672b', 'request_id': 'req-c0158d88-1ad6-4a2e-8fe6-66b13f0a350d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:04.231261 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-c0158d88-1ad6-4a2e-8fe6-66b13f0a350d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '134860c025634d3f9c8896d1456e672b', 'request_id': 'req-c0158d88-1ad6-4a2e-8fe6-66b13f0a350d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:04.318307 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-c0158d88-1ad6-4a2e-8fe6-66b13f0a350d admin admin] The token cache is being invalidate because project 134860c025634d3f9c8896d1456e672b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:35:04.318599 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-c0158d88-1ad6-4a2e-8fe6-66b13f0a350d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:04.319211 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-c0158d88-1ad6-4a2e-8fe6-66b13f0a350d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:04.320839 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1069/2163] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:35:03 2022] DELETE /identity/v3/projects/134860c025634d3f9c8896d1456e672b => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:35:04.325047 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7fb88-1e65-43de-8df3-cd2c37d3d1e5 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:04.326648 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-01c7fb88-1e65-43de-8df3-cd2c37d3d1e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:04.340648 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7fb88-1e65-43de-8df3-cd2c37d3d1e5 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:04.406993 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf262bc-91e3-4dbc-820a-e190c8100d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:04.408068 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf262bc-91e3-4dbc-820a-e190c8100d72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:04.408458 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf262bc-91e3-4dbc-820a-e190c8100d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:04.408807 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf262bc-91e3-4dbc-820a-e190c8100d72 admin admin] PATH_INFO: `/v3/users/1f6f3a9933954aedb068cdad4d0725af` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:04.410527 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf262bc-91e3-4dbc-820a-e190c8100d72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f6f3a9933954aedb068cdad4d0725af)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:04.412730 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf262bc-91e3-4dbc-820a-e190c8100d72 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:04.610718 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:35:04.612304 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-2bf262bc-91e3-4dbc-820a-e190c8100d72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f6f3a9933954aedb068cdad4d0725af'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:04.613681 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-2bf262bc-91e3-4dbc-820a-e190c8100d72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f6f3a9933954aedb068cdad4d0725af'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:04.622120 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-2bf262bc-91e3-4dbc-820a-e190c8100d72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f6f3a9933954aedb068cdad4d0725af'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:04.622811 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-2bf262bc-91e3-4dbc-820a-e190c8100d72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f6f3a9933954aedb068cdad4d0725af'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:04.636326 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-2bf262bc-91e3-4dbc-820a-e190c8100d72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f6f3a9933954aedb068cdad4d0725af'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:04.740713 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1095/2164] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:35:04 2022] DELETE /identity/v3/users/1f6f3a9933954aedb068cdad4d0725af => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:35:05.202739 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0158d88-1ad6-4a2e-8fe6-66b13f0a350d admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:05.204208 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-c0158d88-1ad6-4a2e-8fe6-66b13f0a350d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:05.218055 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0158d88-1ad6-4a2e-8fe6-66b13f0a350d admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:05.277934 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6056321-4195-4d60-920b-56deea812161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:05.278858 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6056321-4195-4d60-920b-56deea812161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:05.279698 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6056321-4195-4d60-920b-56deea812161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:05.279998 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6056321-4195-4d60-920b-56deea812161 admin admin] PATH_INFO: `/v3/projects/7c3bd3b536ae4b509505e962f87c343c` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:05.281198 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6056321-4195-4d60-920b-56deea812161 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c3bd3b536ae4b509505e962f87c343c)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:05.283296 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6056321-4195-4d60-920b-56deea812161 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:05.395630 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-a6056321-4195-4d60-920b-56deea812161 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c3bd3b536ae4b509505e962f87c343c', 'request_id': 'req-a6056321-4195-4d60-920b-56deea812161'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:05.401908 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-a6056321-4195-4d60-920b-56deea812161 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c3bd3b536ae4b509505e962f87c343c', 'request_id': 'req-a6056321-4195-4d60-920b-56deea812161'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:05.408819 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-a6056321-4195-4d60-920b-56deea812161 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c3bd3b536ae4b509505e962f87c343c', 'request_id': 'req-a6056321-4195-4d60-920b-56deea812161'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:05.481202 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-a6056321-4195-4d60-920b-56deea812161 admin admin] The token cache is being invalidate because project 7c3bd3b536ae4b509505e962f87c343c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:35:05.481503 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-a6056321-4195-4d60-920b-56deea812161 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:05.482113 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-a6056321-4195-4d60-920b-56deea812161 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:05.484328 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1070/2165] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:35:05 2022] DELETE /identity/v3/projects/7c3bd3b536ae4b509505e962f87c343c => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:35:08.718385 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf262bc-91e3-4dbc-820a-e190c8100d72 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:08.720592 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-2bf262bc-91e3-4dbc-820a-e190c8100d72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:08.734802 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf262bc-91e3-4dbc-820a-e190c8100d72 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:08.792117 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f51156-62fd-4867-b8ba-f7037d04ace7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:08.792915 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f51156-62fd-4867-b8ba-f7037d04ace7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:08.793435 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f51156-62fd-4867-b8ba-f7037d04ace7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:08.793435 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f51156-62fd-4867-b8ba-f7037d04ace7 admin admin] PATH_INFO: `/v3/users/a6cbf2a2de3d4021aa890300765d2ded` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:08.795396 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f51156-62fd-4867-b8ba-f7037d04ace7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6cbf2a2de3d4021aa890300765d2ded)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:08.798457 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f51156-62fd-4867-b8ba-f7037d04ace7 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:09.290277 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-70f51156-62fd-4867-b8ba-f7037d04ace7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6cbf2a2de3d4021aa890300765d2ded'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:09.290952 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-70f51156-62fd-4867-b8ba-f7037d04ace7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6cbf2a2de3d4021aa890300765d2ded'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:09.305579 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-70f51156-62fd-4867-b8ba-f7037d04ace7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6cbf2a2de3d4021aa890300765d2ded'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:09.305579 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-70f51156-62fd-4867-b8ba-f7037d04ace7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6cbf2a2de3d4021aa890300765d2ded'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:09.317949 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-70f51156-62fd-4867-b8ba-f7037d04ace7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6cbf2a2de3d4021aa890300765d2ded'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:09.413323 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1096/2166] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:35:08 2022] DELETE /identity/v3/users/a6cbf2a2de3d4021aa890300765d2ded => generated 0 bytes in 696 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:35:09.418229 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c207e-ba3a-420e-8ba5-00903fa04305 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:09.418712 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c207e-ba3a-420e-8ba5-00903fa04305 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:09.418859 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c207e-ba3a-420e-8ba5-00903fa04305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:09.438962 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-c37c207e-ba3a-420e-8ba5-00903fa04305 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:35:09.441212 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-c37c207e-ba3a-420e-8ba5-00903fa04305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:09.450316 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:35:09.500901 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1071/2167] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:35:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:35:09.512489 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f51156-62fd-4867-b8ba-f7037d04ace7 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:09.513951 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-70f51156-62fd-4867-b8ba-f7037d04ace7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:09.559460 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:35:09.571469 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f51156-62fd-4867-b8ba-f7037d04ace7 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:09.575022 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d828ab-de92-4c51-85a9-3908c2f5f93d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:09.575689 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d828ab-de92-4c51-85a9-3908c2f5f93d service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:09.575968 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d828ab-de92-4c51-85a9-3908c2f5f93d service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:09.576240 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d828ab-de92-4c51-85a9-3908c2f5f93d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:09.585734 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d828ab-de92-4c51-85a9-3908c2f5f93d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:09.590336 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d828ab-de92-4c51-85a9-3908c2f5f93d service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:09.597146 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1097/2168] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:35:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:35:10.032962 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37c207e-ba3a-420e-8ba5-00903fa04305 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:10.040949 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-c37c207e-ba3a-420e-8ba5-00903fa04305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:10.054842 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37c207e-ba3a-420e-8ba5-00903fa04305 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:10.059977 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7a8a3c-8599-440f-809b-f6980021ec78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:10.064161 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a8a3c-8599-440f-809b-f6980021ec78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:10.064561 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a8a3c-8599-440f-809b-f6980021ec78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:10.064893 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a8a3c-8599-440f-809b-f6980021ec78 admin admin] PATH_INFO: `/v3/projects/b889917fc9c04bb8b90727f9c83acd82` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:10.066303 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7a8a3c-8599-440f-809b-f6980021ec78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b889917fc9c04bb8b90727f9c83acd82)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:10.068386 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7a8a3c-8599-440f-809b-f6980021ec78 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:10.188093 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-2e7a8a3c-8599-440f-809b-f6980021ec78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b889917fc9c04bb8b90727f9c83acd82', 'request_id': 'req-2e7a8a3c-8599-440f-809b-f6980021ec78'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:10.193513 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-2e7a8a3c-8599-440f-809b-f6980021ec78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b889917fc9c04bb8b90727f9c83acd82', 'request_id': 'req-2e7a8a3c-8599-440f-809b-f6980021ec78'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:10.199629 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-2e7a8a3c-8599-440f-809b-f6980021ec78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b889917fc9c04bb8b90727f9c83acd82', 'request_id': 'req-2e7a8a3c-8599-440f-809b-f6980021ec78'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:10.276573 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-2e7a8a3c-8599-440f-809b-f6980021ec78 admin admin] The token cache is being invalidate because project b889917fc9c04bb8b90727f9c83acd82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:35:10.276573 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-2e7a8a3c-8599-440f-809b-f6980021ec78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:10.276573 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-2e7a8a3c-8599-440f-809b-f6980021ec78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:10.277142 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1072/2169] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:35:10 2022] DELETE /identity/v3/projects/b889917fc9c04bb8b90727f9c83acd82 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:35:10.282139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5f477-b4b0-4040-b2f3-560530976aca None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:10.282668 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5f477-b4b0-4040-b2f3-560530976aca None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:10.282668 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5f477-b4b0-4040-b2f3-560530976aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:10.301605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-0aa5f477-b4b0-4040-b2f3-560530976aca None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:35:10.302815 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-0aa5f477-b4b0-4040-b2f3-560530976aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:10.385585 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1098/2170] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:35:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:35:10.393371 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1aa616-5813-4ec1-b4b7-c144b3ddbe78 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:10.394012 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1aa616-5813-4ec1-b4b7-c144b3ddbe78 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:10.394299 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1aa616-5813-4ec1-b4b7-c144b3ddbe78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:10.420236 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-5b1aa616-5813-4ec1-b4b7-c144b3ddbe78 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:35:10.421760 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-5b1aa616-5813-4ec1-b4b7-c144b3ddbe78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:10.432102 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1073/2171] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:35:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:35:10.436227 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa5f477-b4b0-4040-b2f3-560530976aca None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:10.446249 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa5f477-b4b0-4040-b2f3-560530976aca None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:10.459848 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6ee3f-07d4-45d0-942c-725b3e58c20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:10.459848 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6ee3f-07d4-45d0-942c-725b3e58c20d admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:10.459848 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6ee3f-07d4-45d0-942c-725b3e58c20d admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:10.459848 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6ee3f-07d4-45d0-942c-725b3e58c20d admin admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:10.460190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf6ee3f-07d4-45d0-942c-725b3e58c20d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:35:10.460190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf6ee3f-07d4-45d0-942c-725b3e58c20d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:10.460190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf6ee3f-07d4-45d0-942c-725b3e58c20d admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:10.470684 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1099/2172] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:35:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:35:10.475042 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1aa616-5813-4ec1-b4b7-c144b3ddbe78 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:10.486060 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1aa616-5813-4ec1-b4b7-c144b3ddbe78 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:10.490848 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f44d8d-76c2-41eb-b722-5616ed2547f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:10.491504 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f44d8d-76c2-41eb-b722-5616ed2547f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:10.491819 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f44d8d-76c2-41eb-b722-5616ed2547f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:10.492101 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f44d8d-76c2-41eb-b722-5616ed2547f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:10.492802 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f44d8d-76c2-41eb-b722-5616ed2547f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:10.494751 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f44d8d-76c2-41eb-b722-5616ed2547f7 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:10.646475 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1074/2173] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:35:10 2022] POST /identity/v3/projects => generated 373 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:35:10.650518 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6ee3f-07d4-45d0-942c-725b3e58c20d admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:10.661032 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6ee3f-07d4-45d0-942c-725b3e58c20d admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:10.727742 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580c39a-09cf-456c-a964-6451b96c7e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:10.728437 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580c39a-09cf-456c-a964-6451b96c7e99 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:10.728723 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580c39a-09cf-456c-a964-6451b96c7e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:10.728999 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580c39a-09cf-456c-a964-6451b96c7e99 admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:10.730269 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d580c39a-09cf-456c-a964-6451b96c7e99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:10.732299 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d580c39a-09cf-456c-a964-6451b96c7e99 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:10.819805 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1100/2174] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:35:10 2022] POST /identity/v3/users => generated 287 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:35:10.821301 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f44d8d-76c2-41eb-b722-5616ed2547f7 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:10.832724 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f44d8d-76c2-41eb-b722-5616ed2547f7 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:10.837520 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b760c-2ad7-4c22-bc95-cd9a03ae822a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:10.838606 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b760c-2ad7-4c22-bc95-cd9a03ae822a admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:10.838885 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b760c-2ad7-4c22-bc95-cd9a03ae822a admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:10.839164 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b760c-2ad7-4c22-bc95-cd9a03ae822a admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:10.839929 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b760c-2ad7-4c22-bc95-cd9a03ae822a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:10.841519 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b760c-2ad7-4c22-bc95-cd9a03ae822a admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:10.858842 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1075/2175] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:35:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:35:10.861631 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580c39a-09cf-456c-a964-6451b96c7e99 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:10.873688 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580c39a-09cf-456c-a964-6451b96c7e99 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:10.878643 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6237eac5-7a38-4908-b5c5-52b4061994df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:10.879336 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237eac5-7a38-4908-b5c5-52b4061994df admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:10.879649 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237eac5-7a38-4908-b5c5-52b4061994df admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:10.879927 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237eac5-7a38-4908-b5c5-52b4061994df admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:10.880691 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6237eac5-7a38-4908-b5c5-52b4061994df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:10.882299 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6237eac5-7a38-4908-b5c5-52b4061994df admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:10.897997 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1101/2176] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:35:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:35:10.898518 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b760c-2ad7-4c22-bc95-cd9a03ae822a admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:10.913740 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b760c-2ad7-4c22-bc95-cd9a03ae822a admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:10.918719 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed5fa3-09cf-4d13-8d59-d64bc775dad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:10.919787 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed5fa3-09cf-4d13-8d59-d64bc775dad6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:10.920131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed5fa3-09cf-4d13-8d59-d64bc775dad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:10.920487 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed5fa3-09cf-4d13-8d59-d64bc775dad6 admin admin] PATH_INFO: `/v3/projects/93a2524947954d9cb1ac11de4f422e88/users/20d859abc92e4e848726790dfe0fc550/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:10.937056 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ed5fa3-09cf-4d13-8d59-d64bc775dad6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93a2524947954d9cb1ac11de4f422e88, user_id=20d859abc92e4e848726790dfe0fc550, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:10.939833 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ed5fa3-09cf-4d13-8d59-d64bc775dad6 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:11.036296 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1076/2177] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:35:10 2022] PUT /identity/v3/projects/93a2524947954d9cb1ac11de4f422e88/users/20d859abc92e4e848726790dfe0fc550/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:35:11.046887 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217690bf-f05d-4f9c-83d0-39b21e4f1863 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:11.047725 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217690bf-f05d-4f9c-83d0-39b21e4f1863 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:11.048013 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217690bf-f05d-4f9c-83d0-39b21e4f1863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:11.075950 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-217690bf-f05d-4f9c-83d0-39b21e4f1863 None None] MFA Rules not processed for user `20d859abc92e4e848726790dfe0fc550`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:35:11.077247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-217690bf-f05d-4f9c-83d0-39b21e4f1863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:11.122537 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:35:11.159490 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1102/2178] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:35:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:35:11.291302 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed5fa3-09cf-4d13-8d59-d64bc775dad6 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:11.293408 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-30ed5fa3-09cf-4d13-8d59-d64bc775dad6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:11.339496 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:35:11.361963 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed5fa3-09cf-4d13-8d59-d64bc775dad6 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:11.365766 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c70022-26b9-4114-b38b-7e470289dbe0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:11.366424 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c70022-26b9-4114-b38b-7e470289dbe0 service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:11.366714 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c70022-26b9-4114-b38b-7e470289dbe0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:11.366992 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c70022-26b9-4114-b38b-7e470289dbe0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:11.376746 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c70022-26b9-4114-b38b-7e470289dbe0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:11.378305 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c70022-26b9-4114-b38b-7e470289dbe0 service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:11.389332 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1077/2179] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:35:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:35:21.320666 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=9 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:35:21.895558 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=10 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:35:22.552577 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912b097-896c-44f5-bcf0-e41395315e22 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:22.553104 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912b097-896c-44f5-bcf0-e41395315e22 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:22.560278 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912b097-896c-44f5-bcf0-e41395315e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:22.570349 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-4912b097-896c-44f5-bcf0-e41395315e22 None None] MFA Rules not processed for user `b2e09539caff4d4698cf29c1ea22d365`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:35:22.571543 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-4912b097-896c-44f5-bcf0-e41395315e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:22.630493 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1103/2180] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:35:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:35:22.895375 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c70022-26b9-4114-b38b-7e470289dbe0 service cinder] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:22.978960 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c70022-26b9-4114-b38b-7e470289dbe0 service cinder] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:22.983994 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9272615-90ae-463a-86fb-51c6703fc888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:22.984998 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9272615-90ae-463a-86fb-51c6703fc888 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:22.985342 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9272615-90ae-463a-86fb-51c6703fc888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:22.985692 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9272615-90ae-463a-86fb-51c6703fc888 admin admin] PATH_INFO: `/v3/users/20d859abc92e4e848726790dfe0fc550` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:22.987028 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9272615-90ae-463a-86fb-51c6703fc888 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20d859abc92e4e848726790dfe0fc550)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:22.990090 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9272615-90ae-463a-86fb-51c6703fc888 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:23.247014 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-b9272615-90ae-463a-86fb-51c6703fc888 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20d859abc92e4e848726790dfe0fc550'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:23.255481 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-b9272615-90ae-463a-86fb-51c6703fc888 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20d859abc92e4e848726790dfe0fc550'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:23.268413 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-b9272615-90ae-463a-86fb-51c6703fc888 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20d859abc92e4e848726790dfe0fc550'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:23.269145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-b9272615-90ae-463a-86fb-51c6703fc888 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20d859abc92e4e848726790dfe0fc550'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:23.340755 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-b9272615-90ae-463a-86fb-51c6703fc888 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20d859abc92e4e848726790dfe0fc550'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:23.347599 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1078/2181] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:35:22 2022] DELETE /identity/v3/users/20d859abc92e4e848726790dfe0fc550 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:35:23.351940 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5d960-1689-43a2-97c5-429cea4ec287 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:23.352460 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5d960-1689-43a2-97c5-429cea4ec287 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:23.352773 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5d960-1689-43a2-97c5-429cea4ec287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:23.370760 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-92f5d960-1689-43a2-97c5-429cea4ec287 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:35:23.372003 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-92f5d960-1689-43a2-97c5-429cea4ec287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:23.436695 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1104/2182] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:35:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:35:23.446660 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9272615-90ae-463a-86fb-51c6703fc888 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:23.448100 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-b9272615-90ae-463a-86fb-51c6703fc888 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:23.510036 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9272615-90ae-463a-86fb-51c6703fc888 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:23.514259 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aaf489-5e3e-4eb5-b656-145278305fb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:23.514921 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aaf489-5e3e-4eb5-b656-145278305fb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:23.515202 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aaf489-5e3e-4eb5-b656-145278305fb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:23.515476 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aaf489-5e3e-4eb5-b656-145278305fb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:23.537264 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aaf489-5e3e-4eb5-b656-145278305fb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:23.539918 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aaf489-5e3e-4eb5-b656-145278305fb2 service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:23.551409 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1079/2183] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:35:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:35:23.999422 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f5d960-1689-43a2-97c5-429cea4ec287 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:24.001050 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-92f5d960-1689-43a2-97c5-429cea4ec287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:24.013234 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f5d960-1689-43a2-97c5-429cea4ec287 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:24.017971 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0284411-7d6d-4869-af0c-d3c6c8e0dbcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:24.018829 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0284411-7d6d-4869-af0c-d3c6c8e0dbcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:24.019113 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0284411-7d6d-4869-af0c-d3c6c8e0dbcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:24.019393 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0284411-7d6d-4869-af0c-d3c6c8e0dbcc admin admin] PATH_INFO: `/v3/projects/93a2524947954d9cb1ac11de4f422e88` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:24.020827 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0284411-7d6d-4869-af0c-d3c6c8e0dbcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=93a2524947954d9cb1ac11de4f422e88)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:24.022759 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0284411-7d6d-4869-af0c-d3c6c8e0dbcc admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:24.164516 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-a0284411-7d6d-4869-af0c-d3c6c8e0dbcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93a2524947954d9cb1ac11de4f422e88', 'request_id': 'req-a0284411-7d6d-4869-af0c-d3c6c8e0dbcc'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:24.170630 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-a0284411-7d6d-4869-af0c-d3c6c8e0dbcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93a2524947954d9cb1ac11de4f422e88', 'request_id': 'req-a0284411-7d6d-4869-af0c-d3c6c8e0dbcc'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:24.177536 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-a0284411-7d6d-4869-af0c-d3c6c8e0dbcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93a2524947954d9cb1ac11de4f422e88', 'request_id': 'req-a0284411-7d6d-4869-af0c-d3c6c8e0dbcc'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:24.275332 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-a0284411-7d6d-4869-af0c-d3c6c8e0dbcc admin admin] The token cache is being invalidate because project 93a2524947954d9cb1ac11de4f422e88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:35:24.275750 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-a0284411-7d6d-4869-af0c-d3c6c8e0dbcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:24.276641 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-a0284411-7d6d-4869-af0c-d3c6c8e0dbcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:24.279753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1105/2184] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:35:23 2022] DELETE /identity/v3/projects/93a2524947954d9cb1ac11de4f422e88 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:35:33.546238 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:35:34.271694 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:35:42.182503 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a08ea-287a-49e8-9d42-65dace623bd9 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:42.183051 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a08ea-287a-49e8-9d42-65dace623bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:42.183142 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a08ea-287a-49e8-9d42-65dace623bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:42.240734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-e67a08ea-287a-49e8-9d42-65dace623bd9 None None] MFA Rules not processed for user `cd3c04f39d774998b66e6ce920925691`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:35:42.240734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-e67a08ea-287a-49e8-9d42-65dace623bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:42.339323 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1080/2185] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Jul 7 15:35:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:35:42.414455 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0284411-7d6d-4869-af0c-d3c6c8e0dbcc admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:42.417152 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-a0284411-7d6d-4869-af0c-d3c6c8e0dbcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:42.583468 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0284411-7d6d-4869-af0c-d3c6c8e0dbcc admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:42.588805 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27075a2-3215-436c-8744-80f13682a204 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b18033f849b240999d12c1885d2f7b91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:42.589552 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27075a2-3215-436c-8744-80f13682a204 service swift] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:42.589905 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27075a2-3215-436c-8744-80f13682a204 service swift] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:42.590188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27075a2-3215-436c-8744-80f13682a204 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:42.599701 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27075a2-3215-436c-8744-80f13682a204 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:42.601381 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27075a2-3215-436c-8744-80f13682a204 service swift] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:42.610053 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1106/2186] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Jul 7 15:35:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 07 15:35:45.486123 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a08ea-287a-49e8-9d42-65dace623bd9 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:45.488505 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-e67a08ea-287a-49e8-9d42-65dace623bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:45.660449 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a08ea-287a-49e8-9d42-65dace623bd9 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:45.673338 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd1ee35-845a-42d9-a8fd-751f532f75c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:45.674320 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1ee35-845a-42d9-a8fd-751f532f75c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:45.674684 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1ee35-845a-42d9-a8fd-751f532f75c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:45.675155 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1ee35-845a-42d9-a8fd-751f532f75c5 admin admin] PATH_INFO: `/v3/users/70590fe3a00a4d56bcb61f99f52c86e8` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:45.677028 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd1ee35-845a-42d9-a8fd-751f532f75c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70590fe3a00a4d56bcb61f99f52c86e8)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:45.679262 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd1ee35-845a-42d9-a8fd-751f532f75c5 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:45.867963 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:35:45.877980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-3dd1ee35-845a-42d9-a8fd-751f532f75c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70590fe3a00a4d56bcb61f99f52c86e8'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:45.886087 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-3dd1ee35-845a-42d9-a8fd-751f532f75c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70590fe3a00a4d56bcb61f99f52c86e8'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:45.903555 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-3dd1ee35-845a-42d9-a8fd-751f532f75c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70590fe3a00a4d56bcb61f99f52c86e8'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:45.904321 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-3dd1ee35-845a-42d9-a8fd-751f532f75c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70590fe3a00a4d56bcb61f99f52c86e8'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:45.982811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-3dd1ee35-845a-42d9-a8fd-751f532f75c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70590fe3a00a4d56bcb61f99f52c86e8'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:45.994385 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1081/2187] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:35:45 2022] DELETE /identity/v3/users/70590fe3a00a4d56bcb61f99f52c86e8 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:35:45.999859 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892de6f7-b41a-4f21-97fb-8ef263a9cb19 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:46.000791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892de6f7-b41a-4f21-97fb-8ef263a9cb19 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:46.002061 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892de6f7-b41a-4f21-97fb-8ef263a9cb19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:46.036081 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-892de6f7-b41a-4f21-97fb-8ef263a9cb19 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:35:46.038745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-892de6f7-b41a-4f21-97fb-8ef263a9cb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:46.134607 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1107/2188] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:35:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:35:46.161274 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd1ee35-845a-42d9-a8fd-751f532f75c5 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:46.161274 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-3dd1ee35-845a-42d9-a8fd-751f532f75c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:46.311428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd1ee35-845a-42d9-a8fd-751f532f75c5 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:46.311428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f607985-ca5b-4599-b15e-c9f1821fea9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:46.311428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f607985-ca5b-4599-b15e-c9f1821fea9a service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:46.311428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f607985-ca5b-4599-b15e-c9f1821fea9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:46.312259 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f607985-ca5b-4599-b15e-c9f1821fea9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:46.329087 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f607985-ca5b-4599-b15e-c9f1821fea9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:46.330666 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f607985-ca5b-4599-b15e-c9f1821fea9a service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:46.342041 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1082/2189] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:35:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:35:47.182914 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892de6f7-b41a-4f21-97fb-8ef263a9cb19 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:47.186658 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-892de6f7-b41a-4f21-97fb-8ef263a9cb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:47.205026 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892de6f7-b41a-4f21-97fb-8ef263a9cb19 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:47.221293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f331c9-72eb-4bf9-9c2e-55ff397df92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:47.222622 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f331c9-72eb-4bf9-9c2e-55ff397df92e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:47.223069 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f331c9-72eb-4bf9-9c2e-55ff397df92e admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:47.224038 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f331c9-72eb-4bf9-9c2e-55ff397df92e admin admin] PATH_INFO: `/v3/projects/4c9f51214c284f47a318c428797e4bcc` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:47.235144 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f331c9-72eb-4bf9-9c2e-55ff397df92e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c9f51214c284f47a318c428797e4bcc)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:47.238490 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f331c9-72eb-4bf9-9c2e-55ff397df92e admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:47.394538 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-b0f331c9-72eb-4bf9-9c2e-55ff397df92e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c9f51214c284f47a318c428797e4bcc', 'request_id': 'req-b0f331c9-72eb-4bf9-9c2e-55ff397df92e'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:47.404247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:35:47.415596 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-b0f331c9-72eb-4bf9-9c2e-55ff397df92e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c9f51214c284f47a318c428797e4bcc', 'request_id': 'req-b0f331c9-72eb-4bf9-9c2e-55ff397df92e'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:47.431909 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-b0f331c9-72eb-4bf9-9c2e-55ff397df92e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c9f51214c284f47a318c428797e4bcc', 'request_id': 'req-b0f331c9-72eb-4bf9-9c2e-55ff397df92e'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:47.552587 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-b0f331c9-72eb-4bf9-9c2e-55ff397df92e admin admin] The token cache is being invalidate because project 4c9f51214c284f47a318c428797e4bcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:35:47.553986 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-b0f331c9-72eb-4bf9-9c2e-55ff397df92e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:47.555917 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-b0f331c9-72eb-4bf9-9c2e-55ff397df92e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:47.563908 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1108/2190] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:35:47 2022] DELETE /identity/v3/projects/4c9f51214c284f47a318c428797e4bcc => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:35:47.568925 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f698d605-115f-455e-b633-c5ea1b0a1011 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:47.569418 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f698d605-115f-455e-b633-c5ea1b0a1011 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:47.569533 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f698d605-115f-455e-b633-c5ea1b0a1011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:47.700259 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-f698d605-115f-455e-b633-c5ea1b0a1011 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:35:47.704480 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-f698d605-115f-455e-b633-c5ea1b0a1011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:47.933087 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1083/2191] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:35:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:35:47.934867 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72879b89-41c9-4c9d-b06e-552074be616e None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:47.935176 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72879b89-41c9-4c9d-b06e-552074be616e None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:47.935551 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72879b89-41c9-4c9d-b06e-552074be616e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:48.005405 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-72879b89-41c9-4c9d-b06e-552074be616e None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:35:48.007271 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-72879b89-41c9-4c9d-b06e-552074be616e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:48.035456 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1109/2192] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:35:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:35:48.036902 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f698d605-115f-455e-b633-c5ea1b0a1011 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:48.049513 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f698d605-115f-455e-b633-c5ea1b0a1011 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:48.065223 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c03dc-cf8f-4dfe-ba77-758b557a8be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:48.066291 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c03dc-cf8f-4dfe-ba77-758b557a8be0 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:48.066763 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c03dc-cf8f-4dfe-ba77-758b557a8be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:48.068709 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c03dc-cf8f-4dfe-ba77-758b557a8be0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:48.073672 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c03dc-cf8f-4dfe-ba77-758b557a8be0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:35:48.073672 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c03dc-cf8f-4dfe-ba77-758b557a8be0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:48.075489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c03dc-cf8f-4dfe-ba77-758b557a8be0 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:48.136981 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1084/2193] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:35:48.141106 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72879b89-41c9-4c9d-b06e-552074be616e None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:48.153217 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72879b89-41c9-4c9d-b06e-552074be616e None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:48.166708 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea8aef3-9cdc-4004-8aa0-59d1257dbf9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:48.167390 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8aef3-9cdc-4004-8aa0-59d1257dbf9e admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:48.168324 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8aef3-9cdc-4004-8aa0-59d1257dbf9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:48.168324 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8aef3-9cdc-4004-8aa0-59d1257dbf9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:48.168686 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea8aef3-9cdc-4004-8aa0-59d1257dbf9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:48.171015 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea8aef3-9cdc-4004-8aa0-59d1257dbf9e admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:48.318003 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1110/2194] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:35:48 2022] POST /identity/v3/projects => generated 371 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:35:48.322848 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c03dc-cf8f-4dfe-ba77-758b557a8be0 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:48.333666 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:35:48.342203 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c03dc-cf8f-4dfe-ba77-758b557a8be0 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:48.428910 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4899ea86-c6d0-42d8-b0d9-de4f44091707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:48.430317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899ea86-c6d0-42d8-b0d9-de4f44091707 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:48.430715 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899ea86-c6d0-42d8-b0d9-de4f44091707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:48.431054 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899ea86-c6d0-42d8-b0d9-de4f44091707 admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:48.431953 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4899ea86-c6d0-42d8-b0d9-de4f44091707 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:48.434161 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4899ea86-c6d0-42d8-b0d9-de4f44091707 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:48.557244 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea8aef3-9cdc-4004-8aa0-59d1257dbf9e admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:48.557710 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1085/2195] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:35:48 2022] POST /identity/v3/users => generated 286 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:35:48.573659 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea8aef3-9cdc-4004-8aa0-59d1257dbf9e admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:48.579220 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9429d-3af8-4d82-a7a5-981a4866ca2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:48.580009 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9429d-3af8-4d82-a7a5-981a4866ca2e admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:48.580302 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9429d-3af8-4d82-a7a5-981a4866ca2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:48.580587 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9429d-3af8-4d82-a7a5-981a4866ca2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:48.581324 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f9429d-3af8-4d82-a7a5-981a4866ca2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:48.583588 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f9429d-3af8-4d82-a7a5-981a4866ca2e admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:48.599643 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1111/2196] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:35:48 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:35:48.602593 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4899ea86-c6d0-42d8-b0d9-de4f44091707 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:48.632675 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4899ea86-c6d0-42d8-b0d9-de4f44091707 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:48.640014 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e99baad-3d86-490c-9a28-91845f2c411b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:48.642865 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e99baad-3d86-490c-9a28-91845f2c411b admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:48.642865 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e99baad-3d86-490c-9a28-91845f2c411b admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:48.642865 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e99baad-3d86-490c-9a28-91845f2c411b admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:48.642865 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e99baad-3d86-490c-9a28-91845f2c411b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:48.643945 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e99baad-3d86-490c-9a28-91845f2c411b admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:48.662596 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1086/2197] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:35:48 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:35:48.663834 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9429d-3af8-4d82-a7a5-981a4866ca2e admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:48.675854 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9429d-3af8-4d82-a7a5-981a4866ca2e admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:48.681672 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd10347-f244-4c15-8bee-b27b156fec31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:48.682582 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd10347-f244-4c15-8bee-b27b156fec31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:48.682862 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd10347-f244-4c15-8bee-b27b156fec31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:48.683138 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd10347-f244-4c15-8bee-b27b156fec31 admin admin] PATH_INFO: `/v3/projects/41be827d61314cdbb5387c24e6353bb5/users/4a7231c86e1a4c4bb851f317a935079d/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:48.705210 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd10347-f244-4c15-8bee-b27b156fec31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41be827d61314cdbb5387c24e6353bb5, user_id=4a7231c86e1a4c4bb851f317a935079d, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:48.707896 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd10347-f244-4c15-8bee-b27b156fec31 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:48.812291 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1112/2198] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:35:48 2022] PUT /identity/v3/projects/41be827d61314cdbb5387c24e6353bb5/users/4a7231c86e1a4c4bb851f317a935079d/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:35:48.817696 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84efe715-f563-4487-958c-4884714b763d None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:48.817993 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84efe715-f563-4487-958c-4884714b763d None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:48.818336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84efe715-f563-4487-958c-4884714b763d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:48.848554 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-84efe715-f563-4487-958c-4884714b763d None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:35:48.855348 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-84efe715-f563-4487-958c-4884714b763d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:48.980984 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1087/2199] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:35:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:35:48.991390 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd10347-f244-4c15-8bee-b27b156fec31 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:48.993509 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-9fd10347-f244-4c15-8bee-b27b156fec31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:49.086441 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd10347-f244-4c15-8bee-b27b156fec31 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:49.090154 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d9937-b50d-413f-a979-d2d03e73030b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:49.090869 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d9937-b50d-413f-a979-d2d03e73030b service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:49.091219 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d9937-b50d-413f-a979-d2d03e73030b service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:49.091612 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d9937-b50d-413f-a979-d2d03e73030b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:49.102184 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3d9937-b50d-413f-a979-d2d03e73030b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:49.105416 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3d9937-b50d-413f-a979-d2d03e73030b service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:49.117620 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1113/2200] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:35:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:35:49.143485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84efe715-f563-4487-958c-4884714b763d None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:49.145566 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-84efe715-f563-4487-958c-4884714b763d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:49.176502 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84efe715-f563-4487-958c-4884714b763d None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:49.189222 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74bc613-3846-4f52-b136-6f3a2cbeb2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:49.190117 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74bc613-3846-4f52-b136-6f3a2cbeb2fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:49.190401 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74bc613-3846-4f52-b136-6f3a2cbeb2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:49.190680 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74bc613-3846-4f52-b136-6f3a2cbeb2fe admin admin] PATH_INFO: `/v3/users/9bb4a2d7b0584fcdaaf296b31ad91ab6` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:49.194485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74bc613-3846-4f52-b136-6f3a2cbeb2fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bb4a2d7b0584fcdaaf296b31ad91ab6)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:49.196950 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74bc613-3846-4f52-b136-6f3a2cbeb2fe admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:49.431489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-f74bc613-3846-4f52-b136-6f3a2cbeb2fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bb4a2d7b0584fcdaaf296b31ad91ab6'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:49.452922 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-f74bc613-3846-4f52-b136-6f3a2cbeb2fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bb4a2d7b0584fcdaaf296b31ad91ab6'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:49.481061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-f74bc613-3846-4f52-b136-6f3a2cbeb2fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bb4a2d7b0584fcdaaf296b31ad91ab6'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:49.481919 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-f74bc613-3846-4f52-b136-6f3a2cbeb2fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bb4a2d7b0584fcdaaf296b31ad91ab6'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:49.486928 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:35:49.608838 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-f74bc613-3846-4f52-b136-6f3a2cbeb2fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bb4a2d7b0584fcdaaf296b31ad91ab6'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:49.620901 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1088/2201] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:35:49 2022] DELETE /identity/v3/users/9bb4a2d7b0584fcdaaf296b31ad91ab6 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:35:49.623257 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c87b60-ea1f-4ae7-b874-78822248f22e None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:49.624982 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c87b60-ea1f-4ae7-b874-78822248f22e None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:49.625433 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c87b60-ea1f-4ae7-b874-78822248f22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:49.646417 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-a7c87b60-ea1f-4ae7-b874-78822248f22e None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:35:49.647831 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-a7c87b60-ea1f-4ae7-b874-78822248f22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:49.773759 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1114/2202] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:35:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:35:49.780740 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74bc613-3846-4f52-b136-6f3a2cbeb2fe admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:49.782759 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-f74bc613-3846-4f52-b136-6f3a2cbeb2fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:49.871948 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74bc613-3846-4f52-b136-6f3a2cbeb2fe admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:49.876700 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16dcb57-9c69-4dc1-8b9b-c51074e42290 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:49.877024 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16dcb57-9c69-4dc1-8b9b-c51074e42290 service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:49.877406 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16dcb57-9c69-4dc1-8b9b-c51074e42290 service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:49.877781 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16dcb57-9c69-4dc1-8b9b-c51074e42290 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:49.888603 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16dcb57-9c69-4dc1-8b9b-c51074e42290 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:49.890220 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16dcb57-9c69-4dc1-8b9b-c51074e42290 service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:49.914059 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1089/2203] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:35:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:35:50.829434 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c87b60-ea1f-4ae7-b874-78822248f22e None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:50.829434 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-a7c87b60-ea1f-4ae7-b874-78822248f22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:50.862027 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c87b60-ea1f-4ae7-b874-78822248f22e None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:50.867878 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62281128-2dec-45ac-a890-6976289cab97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:50.868742 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62281128-2dec-45ac-a890-6976289cab97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:50.869021 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62281128-2dec-45ac-a890-6976289cab97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:50.869294 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62281128-2dec-45ac-a890-6976289cab97 admin admin] PATH_INFO: `/v3/projects/7d6edec41e0049b5903c78d2a77620dc` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:50.870575 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62281128-2dec-45ac-a890-6976289cab97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d6edec41e0049b5903c78d2a77620dc)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:50.873046 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62281128-2dec-45ac-a890-6976289cab97 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:50.895524 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:35:51.019650 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-62281128-2dec-45ac-a890-6976289cab97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d6edec41e0049b5903c78d2a77620dc', 'request_id': 'req-62281128-2dec-45ac-a890-6976289cab97'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:51.025898 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-62281128-2dec-45ac-a890-6976289cab97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d6edec41e0049b5903c78d2a77620dc', 'request_id': 'req-62281128-2dec-45ac-a890-6976289cab97'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:51.033259 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-62281128-2dec-45ac-a890-6976289cab97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d6edec41e0049b5903c78d2a77620dc', 'request_id': 'req-62281128-2dec-45ac-a890-6976289cab97'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:51.120431 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-62281128-2dec-45ac-a890-6976289cab97 admin admin] The token cache is being invalidate because project 7d6edec41e0049b5903c78d2a77620dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:35:51.122609 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-62281128-2dec-45ac-a890-6976289cab97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:51.122609 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-62281128-2dec-45ac-a890-6976289cab97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:35:51.123247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1115/2204] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:35:50 2022] DELETE /identity/v3/projects/7d6edec41e0049b5903c78d2a77620dc => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:35:51.131540 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95521acc-9913-46db-a3c3-54153120da5f None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:51.132144 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95521acc-9913-46db-a3c3-54153120da5f None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:51.132347 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95521acc-9913-46db-a3c3-54153120da5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:51.154284 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-95521acc-9913-46db-a3c3-54153120da5f None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:35:51.155831 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-95521acc-9913-46db-a3c3-54153120da5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:51.253356 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1090/2205] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:35:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:35:51.254356 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e83b43-6634-483c-bc72-1ec03704462f None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:51.254714 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e83b43-6634-483c-bc72-1ec03704462f None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:51.255063 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e83b43-6634-483c-bc72-1ec03704462f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:51.282314 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-69e83b43-6634-483c-bc72-1ec03704462f None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:35:51.284296 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-69e83b43-6634-483c-bc72-1ec03704462f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:51.297117 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1116/2206] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:35:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:35:51.305397 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95521acc-9913-46db-a3c3-54153120da5f None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:51.324834 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95521acc-9913-46db-a3c3-54153120da5f None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:51.335174 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546f117-7946-4de6-95c5-bd49deef80f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:51.337247 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546f117-7946-4de6-95c5-bd49deef80f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:51.340165 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546f117-7946-4de6-95c5-bd49deef80f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:51.340165 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546f117-7946-4de6-95c5-bd49deef80f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:51.340165 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2546f117-7946-4de6-95c5-bd49deef80f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:35:51.340165 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2546f117-7946-4de6-95c5-bd49deef80f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:51.341548 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2546f117-7946-4de6-95c5-bd49deef80f2 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:51.369012 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e83b43-6634-483c-bc72-1ec03704462f None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:51.380885 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1091/2207] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:35:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:35:51.386141 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e83b43-6634-483c-bc72-1ec03704462f None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:51.392600 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674da018-f4f1-43dd-8b04-d176fab728b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:51.393364 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674da018-f4f1-43dd-8b04-d176fab728b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:51.393712 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674da018-f4f1-43dd-8b04-d176fab728b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:51.394163 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674da018-f4f1-43dd-8b04-d176fab728b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:51.401665 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674da018-f4f1-43dd-8b04-d176fab728b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:51.404922 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674da018-f4f1-43dd-8b04-d176fab728b8 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:51.509229 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1117/2208] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:35:51 2022] POST /identity/v3/projects => generated 373 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:35:51.514860 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546f117-7946-4de6-95c5-bd49deef80f2 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:51.538042 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546f117-7946-4de6-95c5-bd49deef80f2 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:51.663979 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58f899c-c183-460c-8b4d-11bd9c39eb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:51.666741 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f899c-c183-460c-8b4d-11bd9c39eb06 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:51.666741 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f899c-c183-460c-8b4d-11bd9c39eb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:51.666741 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f899c-c183-460c-8b4d-11bd9c39eb06 admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:51.669847 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58f899c-c183-460c-8b4d-11bd9c39eb06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:51.675162 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58f899c-c183-460c-8b4d-11bd9c39eb06 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:51.798834 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674da018-f4f1-43dd-8b04-d176fab728b8 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:51.806224 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1092/2209] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:35:51 2022] POST /identity/v3/users => generated 287 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:35:51.811960 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674da018-f4f1-43dd-8b04-d176fab728b8 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:51.827443 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63cf2fe-b5ef-464e-b649-109a645c8ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:51.828325 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cf2fe-b5ef-464e-b649-109a645c8ec4 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:51.828809 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cf2fe-b5ef-464e-b649-109a645c8ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:51.829358 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cf2fe-b5ef-464e-b649-109a645c8ec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:51.840743 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63cf2fe-b5ef-464e-b649-109a645c8ec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:51.844032 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63cf2fe-b5ef-464e-b649-109a645c8ec4 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:51.865893 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58f899c-c183-460c-8b4d-11bd9c39eb06 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:51.878884 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1118/2210] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:35:51 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:35:51.887426 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58f899c-c183-460c-8b4d-11bd9c39eb06 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:51.904215 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d3d183-0ae8-49f1-8e4a-878b4635efa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:51.904901 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3d183-0ae8-49f1-8e4a-878b4635efa7 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:51.905186 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3d183-0ae8-49f1-8e4a-878b4635efa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:51.905526 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3d183-0ae8-49f1-8e4a-878b4635efa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:51.906266 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d3d183-0ae8-49f1-8e4a-878b4635efa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:51.908443 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d3d183-0ae8-49f1-8e4a-878b4635efa7 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:51.944348 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1093/2211] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:35:51 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:35:51.948384 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63cf2fe-b5ef-464e-b649-109a645c8ec4 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:51.978121 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63cf2fe-b5ef-464e-b649-109a645c8ec4 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:51.995295 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e686f7-4333-4b63-b948-8eff492ccebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:51.996241 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e686f7-4333-4b63-b948-8eff492ccebd admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:51.996521 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e686f7-4333-4b63-b948-8eff492ccebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:51.996795 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e686f7-4333-4b63-b948-8eff492ccebd admin admin] PATH_INFO: `/v3/projects/b42fb18bea524ba980012973ec2a9ee2/users/3256d562f0a64600baeb215ebff4190d/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:52.050057 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e686f7-4333-4b63-b948-8eff492ccebd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b42fb18bea524ba980012973ec2a9ee2, user_id=3256d562f0a64600baeb215ebff4190d, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:52.053160 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e686f7-4333-4b63-b948-8eff492ccebd admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:52.141338 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1119/2212] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:35:51 2022] PUT /identity/v3/projects/b42fb18bea524ba980012973ec2a9ee2/users/3256d562f0a64600baeb215ebff4190d/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:35:52.154515 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1fa349-9d7f-4414-ac8f-ccc8ef089728 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:52.155226 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1fa349-9d7f-4414-ac8f-ccc8ef089728 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:52.155781 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1fa349-9d7f-4414-ac8f-ccc8ef089728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:52.164755 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:35:52.207822 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-3e1fa349-9d7f-4414-ac8f-ccc8ef089728 None None] MFA Rules not processed for user `3256d562f0a64600baeb215ebff4190d`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:35:52.209375 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-3e1fa349-9d7f-4414-ac8f-ccc8ef089728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:52.341305 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1094/2213] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:35:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:35:52.368774 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e686f7-4333-4b63-b948-8eff492ccebd admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:52.372611 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-70e686f7-4333-4b63-b948-8eff492ccebd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:52.457053 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e686f7-4333-4b63-b948-8eff492ccebd admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:52.462388 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ebe3ad-13ab-4a62-aeb2-064acc862eb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:52.462981 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebe3ad-13ab-4a62-aeb2-064acc862eb4 service cinder] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:52.464778 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebe3ad-13ab-4a62-aeb2-064acc862eb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:52.465311 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebe3ad-13ab-4a62-aeb2-064acc862eb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:52.491749 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ebe3ad-13ab-4a62-aeb2-064acc862eb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:52.493616 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ebe3ad-13ab-4a62-aeb2-064acc862eb4 service cinder] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:52.534610 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1120/2214] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:35:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:35:54.595022 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848bb76e-0a9a-478c-a74d-c78f5ea03754 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:54.596356 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848bb76e-0a9a-478c-a74d-c78f5ea03754 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:54.596812 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848bb76e-0a9a-478c-a74d-c78f5ea03754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:54.645067 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-848bb76e-0a9a-478c-a74d-c78f5ea03754 None None] MFA Rules not processed for user `3256d562f0a64600baeb215ebff4190d`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:35:54.650571 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-848bb76e-0a9a-478c-a74d-c78f5ea03754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:54.675106 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1095/2215] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:35:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:35:59.294046 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83dade-1b35-4d52-b56c-dc0ed9a3f558 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:59.294594 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83dade-1b35-4d52-b56c-dc0ed9a3f558 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:59.294787 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83dade-1b35-4d52-b56c-dc0ed9a3f558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:59.393526 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-6c83dade-1b35-4d52-b56c-dc0ed9a3f558 None None] MFA Rules not processed for user `4a7231c86e1a4c4bb851f317a935079d`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:35:59.395190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-6c83dade-1b35-4d52-b56c-dc0ed9a3f558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:59.475606 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:35:59.513792 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1121/2216] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:35:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:35:59.518969 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848bb76e-0a9a-478c-a74d-c78f5ea03754 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:59.522267 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-848bb76e-0a9a-478c-a74d-c78f5ea03754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:35:59.601309 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848bb76e-0a9a-478c-a74d-c78f5ea03754 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:59.606913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b93697-da52-4ba8-960a-3acce5a2f188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:59.607647 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b93697-da52-4ba8-960a-3acce5a2f188 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:59.607894 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b93697-da52-4ba8-960a-3acce5a2f188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:59.608588 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b93697-da52-4ba8-960a-3acce5a2f188 admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:59.609311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b93697-da52-4ba8-960a-3acce5a2f188 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:59.611524 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b93697-da52-4ba8-960a-3acce5a2f188 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:35:59.747342 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1096/2217] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:35:59 2022] POST /identity/v3/projects => generated 371 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:35:59.748155 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c83dade-1b35-4d52-b56c-dc0ed9a3f558 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:35:59.772850 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c83dade-1b35-4d52-b56c-dc0ed9a3f558 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:35:59.905901 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557515b2-7182-49f8-9b0b-70f6547f58db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:35:59.908718 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557515b2-7182-49f8-9b0b-70f6547f58db admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:35:59.909164 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557515b2-7182-49f8-9b0b-70f6547f58db admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:35:59.909523 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557515b2-7182-49f8-9b0b-70f6547f58db admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:35:59.910376 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557515b2-7182-49f8-9b0b-70f6547f58db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:35:59.913224 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557515b2-7182-49f8-9b0b-70f6547f58db admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:36:00.002612 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1122/2218] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:35:59 2022] POST /identity/v3/users => generated 292 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:36:00.008919 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b93697-da52-4ba8-960a-3acce5a2f188 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:36:00.028109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b93697-da52-4ba8-960a-3acce5a2f188 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:36:00.033435 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cdde31-017f-4b76-bbbd-883eb312f1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:36:00.034107 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cdde31-017f-4b76-bbbd-883eb312f1b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:00.034390 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cdde31-017f-4b76-bbbd-883eb312f1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:00.034668 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cdde31-017f-4b76-bbbd-883eb312f1b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:00.035388 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cdde31-017f-4b76-bbbd-883eb312f1b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:36:00.037031 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cdde31-017f-4b76-bbbd-883eb312f1b3 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:36:00.050375 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1097/2219] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:36:00.054389 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557515b2-7182-49f8-9b0b-70f6547f58db admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:36:00.065516 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557515b2-7182-49f8-9b0b-70f6547f58db admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:36:00.071312 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc959e-fae6-41e6-a089-736c5652da4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:36:00.071733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc959e-fae6-41e6-a089-736c5652da4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:00.072099 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc959e-fae6-41e6-a089-736c5652da4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:00.072508 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc959e-fae6-41e6-a089-736c5652da4a admin admin] PATH_INFO: `/v3/projects/7714a76c80c94191a77576e343e64e16/users/17bd76c88a434eb2b79cac3e491430b4/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:00.097654 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc959e-fae6-41e6-a089-736c5652da4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7714a76c80c94191a77576e343e64e16, user_id=17bd76c88a434eb2b79cac3e491430b4, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:36:00.100475 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc959e-fae6-41e6-a089-736c5652da4a admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:36:00.192658 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1123/2220] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:36:00 2022] PUT /identity/v3/projects/7714a76c80c94191a77576e343e64e16/users/17bd76c88a434eb2b79cac3e491430b4/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:36:00.196750 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cdde31-017f-4b76-bbbd-883eb312f1b3 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:36:00.208052 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cdde31-017f-4b76-bbbd-883eb312f1b3 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:36:00.295300 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c00d18-8913-453a-b983-bb19bd75b17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:36:00.296033 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c00d18-8913-453a-b983-bb19bd75b17d admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:00.298864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c00d18-8913-453a-b983-bb19bd75b17d admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:00.298864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c00d18-8913-453a-b983-bb19bd75b17d admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:00.298864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c00d18-8913-453a-b983-bb19bd75b17d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:36:00.299828 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c00d18-8913-453a-b983-bb19bd75b17d admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:36:00.303070 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:36:00.314038 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1098/2221] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:36:00.317672 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc959e-fae6-41e6-a089-736c5652da4a admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:36:00.332496 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc959e-fae6-41e6-a089-736c5652da4a admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:36:00.338278 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae5d5a1-30c4-4fd8-94f0-e84913661087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:36:00.339300 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5d5a1-30c4-4fd8-94f0-e84913661087 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:00.339606 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5d5a1-30c4-4fd8-94f0-e84913661087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:00.339890 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5d5a1-30c4-4fd8-94f0-e84913661087 admin admin] PATH_INFO: `/v3/projects/7714a76c80c94191a77576e343e64e16/users/17bd76c88a434eb2b79cac3e491430b4/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:00.342468 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae5d5a1-30c4-4fd8-94f0-e84913661087 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7714a76c80c94191a77576e343e64e16, user_id=17bd76c88a434eb2b79cac3e491430b4, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:36:00.345979 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae5d5a1-30c4-4fd8-94f0-e84913661087 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:36:00.367983 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1124/2222] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:36:00 2022] PUT /identity/v3/projects/7714a76c80c94191a77576e343e64e16/users/17bd76c88a434eb2b79cac3e491430b4/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:36:05.351711 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee48f6-105d-4b16-b637-cd9bc266c3d7 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:05.352183 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee48f6-105d-4b16-b637-cd9bc266c3d7 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:05.352297 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee48f6-105d-4b16-b637-cd9bc266c3d7 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:05.354197 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1099/2223] 10.222.0.25 () {54 vars in 921 bytes} [Thu Jul 7 15:36:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:36:05.361614 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad024a7a-2da2-47fc-b8aa-d1ab5e997dcb None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:05.362132 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad024a7a-2da2-47fc-b8aa-d1ab5e997dcb None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:05.362229 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad024a7a-2da2-47fc-b8aa-d1ab5e997dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:05.435704 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-ad024a7a-2da2-47fc-b8aa-d1ab5e997dcb None None] MFA Rules not processed for user `de1632555cb345f2b9213da1925c02f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:36:05.436967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-ad024a7a-2da2-47fc-b8aa-d1ab5e997dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:36:05.535685 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:36:05.612843 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1125/2224] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Jul 7 15:36:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:36:05.625748 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee48f6-105d-4b16-b637-cd9bc266c3d7 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:36:05.628617 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-e9ee48f6-105d-4b16-b637-cd9bc266c3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:36:05.702657 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee48f6-105d-4b16-b637-cd9bc266c3d7 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:36:05.709068 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998d4e89-772a-4e0c-87b2-bc80563719f1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0d29e718f446e2a9fd1b2575a86041', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:36:05.719982 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d4e89-772a-4e0c-87b2-bc80563719f1 service glance] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:05.719982 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d4e89-772a-4e0c-87b2-bc80563719f1 service glance] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:05.719982 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d4e89-772a-4e0c-87b2-bc80563719f1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:05.722231 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998d4e89-772a-4e0c-87b2-bc80563719f1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:36:05.723896 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998d4e89-772a-4e0c-87b2-bc80563719f1 service glance] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:36:05.739746 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1100/2225] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:36:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:36:05.968405 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00efd1-70bd-4a8b-b4d5-466328a8efbd None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:05.968805 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00efd1-70bd-4a8b-b4d5-466328a8efbd None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:05.969159 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00efd1-70bd-4a8b-b4d5-466328a8efbd None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:05.971759 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1126/2226] 10.222.0.25 () {54 vars in 921 bytes} [Thu Jul 7 15:36:05 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:36:05.977086 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fd5dc-88a9-4de0-8f9e-c16241162350 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:05.977494 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fd5dc-88a9-4de0-8f9e-c16241162350 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:05.977973 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fd5dc-88a9-4de0-8f9e-c16241162350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:06.010972 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-da9fd5dc-88a9-4de0-8f9e-c16241162350 None None] MFA Rules not processed for user `e79efaea155c46ecb9831956e68aec8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:36:06.012275 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-da9fd5dc-88a9-4de0-8f9e-c16241162350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:36:06.076281 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1101/2227] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Jul 7 15:36:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:36:06.097962 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c00efd1-70bd-4a8b-b4d5-466328a8efbd None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:36:06.102344 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-3c00efd1-70bd-4a8b-b4d5-466328a8efbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:36:06.127651 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c00efd1-70bd-4a8b-b4d5-466328a8efbd None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:36:06.134958 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a07b4-eba9-4710-97e9-6ef629c6f70b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:36:06.135950 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a07b4-eba9-4710-97e9-6ef629c6f70b service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:06.136111 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a07b4-eba9-4710-97e9-6ef629c6f70b service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:06.136477 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a07b4-eba9-4710-97e9-6ef629c6f70b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:06.145460 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a07b4-eba9-4710-97e9-6ef629c6f70b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:36:06.147453 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a07b4-eba9-4710-97e9-6ef629c6f70b service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:36:06.156599 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1127/2228] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:36:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:36:06.302325 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65885aee-3650-480b-adc8-95e459e1d8d2 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:06.302614 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65885aee-3650-480b-adc8-95e459e1d8d2 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:06.302891 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65885aee-3650-480b-adc8-95e459e1d8d2 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:06.304349 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1102/2229] 10.222.0.25 () {54 vars in 921 bytes} [Thu Jul 7 15:36:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:36:06.310483 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c70869-55b9-4441-8282-751dc0c50c84 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:06.311099 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c70869-55b9-4441-8282-751dc0c50c84 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:06.311475 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c70869-55b9-4441-8282-751dc0c50c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:06.332268 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-d7c70869-55b9-4441-8282-751dc0c50c84 None None] MFA Rules not processed for user `de1632555cb345f2b9213da1925c02f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:36:06.333899 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-d7c70869-55b9-4441-8282-751dc0c50c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:36:06.344683 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1128/2230] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Jul 7 15:36:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:36:06.356831 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65885aee-3650-480b-adc8-95e459e1d8d2 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:36:06.372169 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65885aee-3650-480b-adc8-95e459e1d8d2 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:36:06.376062 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af581b08-b152-442f-a11d-b8c553f378b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:36:06.376871 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af581b08-b152-442f-a11d-b8c553f378b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:06.377949 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af581b08-b152-442f-a11d-b8c553f378b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:06.378529 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af581b08-b152-442f-a11d-b8c553f378b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:06.393113 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af581b08-b152-442f-a11d-b8c553f378b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:36:06.394748 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af581b08-b152-442f-a11d-b8c553f378b6 service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:36:06.410208 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1103/2231] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:36:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:36:07.737338 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c70869-55b9-4441-8282-751dc0c50c84 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:36:07.739908 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-d7c70869-55b9-4441-8282-751dc0c50c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:36:08.031739 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c70869-55b9-4441-8282-751dc0c50c84 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:36:08.037550 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e77c86-6e7c-49b1-b384-21233afa164b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '755c2782cfb042f89950b8fc862e4fa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:36:08.038410 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e77c86-6e7c-49b1-b384-21233afa164b service placement] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:08.038791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e77c86-6e7c-49b1-b384-21233afa164b service placement] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:08.039128 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e77c86-6e7c-49b1-b384-21233afa164b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:08.041289 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-32e77c86-6e7c-49b1-b384-21233afa164b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:36:08.057544 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e77c86-6e7c-49b1-b384-21233afa164b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:36:08.059207 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e77c86-6e7c-49b1-b384-21233afa164b service placement] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:36:08.077327 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1129/2232] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:36:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:36:08.376019 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed114f4c-e7bf-4f11-a689-d7b20adf116a None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:08.376568 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed114f4c-e7bf-4f11-a689-d7b20adf116a None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:08.376568 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed114f4c-e7bf-4f11-a689-d7b20adf116a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:08.406507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-ed114f4c-e7bf-4f11-a689-d7b20adf116a None None] MFA Rules not processed for user `17bd76c88a434eb2b79cac3e491430b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:36:08.407893 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-ed114f4c-e7bf-4f11-a689-d7b20adf116a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:36:08.534074 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1104/2233] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:36:08 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:36:08.831913 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e77c86-6e7c-49b1-b384-21233afa164b service placement] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:36:08.836886 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-32e77c86-6e7c-49b1-b384-21233afa164b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:36:08.854583 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e77c86-6e7c-49b1-b384-21233afa164b service placement] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:36:08.860305 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14db9aec-6fdd-410c-a043-08c51424c80f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:36:08.861005 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db9aec-6fdd-410c-a043-08c51424c80f service nova] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:08.861287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db9aec-6fdd-410c-a043-08c51424c80f service nova] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:08.861565 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db9aec-6fdd-410c-a043-08c51424c80f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:08.922505 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14db9aec-6fdd-410c-a043-08c51424c80f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:36:08.924311 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14db9aec-6fdd-410c-a043-08c51424c80f service nova] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:36:08.933729 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1130/2234] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:36:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:36:12.026499 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707eaf16-81e4-4f4a-b5a4-15fb7246e36b None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:12.026960 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707eaf16-81e4-4f4a-b5a4-15fb7246e36b None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:12.027043 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707eaf16-81e4-4f4a-b5a4-15fb7246e36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:12.047368 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-707eaf16-81e4-4f4a-b5a4-15fb7246e36b None None] MFA Rules not processed for user `4a7231c86e1a4c4bb851f317a935079d`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:36:12.049040 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-707eaf16-81e4-4f4a-b5a4-15fb7246e36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:36:12.057649 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1105/2235] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:36:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:36:14.513666 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951e90d-e734-4abc-b7b9-afd37330c1ec None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:14.514161 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951e90d-e734-4abc-b7b9-afd37330c1ec None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:14.514256 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951e90d-e734-4abc-b7b9-afd37330c1ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:14.568496 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:36:14.580508 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-9951e90d-e734-4abc-b7b9-afd37330c1ec None None] MFA Rules not processed for user `cd3c04f39d774998b66e6ce920925691`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:36:14.582023 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-9951e90d-e734-4abc-b7b9-afd37330c1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:36:14.648178 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1131/2236] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Jul 7 15:36:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:36:14.661338 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707eaf16-81e4-4f4a-b5a4-15fb7246e36b None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:36:14.662866 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-707eaf16-81e4-4f4a-b5a4-15fb7246e36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:36:14.682743 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:36:14.758846 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707eaf16-81e4-4f4a-b5a4-15fb7246e36b None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:36:14.764125 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7372187-1f61-4701-a84a-3191df023a0f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b18033f849b240999d12c1885d2f7b91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:36:14.764772 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7372187-1f61-4701-a84a-3191df023a0f service swift] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:14.765058 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7372187-1f61-4701-a84a-3191df023a0f service swift] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:14.765335 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7372187-1f61-4701-a84a-3191df023a0f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:14.777910 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7372187-1f61-4701-a84a-3191df023a0f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:36:14.779514 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7372187-1f61-4701-a84a-3191df023a0f service swift] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:36:14.788610 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1106/2237] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Jul 7 15:36:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 07 15:36:19.471153 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66517a58-4b3f-4d1e-969a-ec86de19099a None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:19.477026 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66517a58-4b3f-4d1e-969a-ec86de19099a None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:19.477026 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66517a58-4b3f-4d1e-969a-ec86de19099a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:19.499603 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-66517a58-4b3f-4d1e-969a-ec86de19099a None None] MFA Rules not processed for user `9528a9a3306d49b0978d77d1ce1ca900`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:36:19.505480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-66517a58-4b3f-4d1e-969a-ec86de19099a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:36:19.581084 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1132/2238] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:36:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:36:24.825161 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=34 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:36:29.571458 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=53 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:36:32.483691 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d5ac0-625a-4142-a7ca-15cdca472b26 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:32.484367 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d5ac0-625a-4142-a7ca-15cdca472b26 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:32.484367 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d5ac0-625a-4142-a7ca-15cdca472b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:32.504957 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-554d5ac0-625a-4142-a7ca-15cdca472b26 None None] MFA Rules not processed for user `cd3c04f39d774998b66e6ce920925691`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:36:32.506236 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-554d5ac0-625a-4142-a7ca-15cdca472b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:36:32.513544 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1107/2239] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Jul 7 15:36:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:36:32.528424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66517a58-4b3f-4d1e-969a-ec86de19099a None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:36:32.542646 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66517a58-4b3f-4d1e-969a-ec86de19099a None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:36:32.548456 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a97e7a-0d0e-4c56-809b-4b3f9a651a6a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b18033f849b240999d12c1885d2f7b91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:36:32.549216 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a97e7a-0d0e-4c56-809b-4b3f9a651a6a service swift] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:32.549552 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a97e7a-0d0e-4c56-809b-4b3f9a651a6a service swift] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:32.549831 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a97e7a-0d0e-4c56-809b-4b3f9a651a6a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:32.559186 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a97e7a-0d0e-4c56-809b-4b3f9a651a6a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:36:32.560961 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a97e7a-0d0e-4c56-809b-4b3f9a651a6a service swift] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:36:32.570147 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1133/2240] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Jul 7 15:36:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 07 15:36:33.490915 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15229053-f4b3-4ba4-8754-57935b197c7c None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:33.493975 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15229053-f4b3-4ba4-8754-57935b197c7c None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:33.493975 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15229053-f4b3-4ba4-8754-57935b197c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:33.510128 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-15229053-f4b3-4ba4-8754-57935b197c7c None None] MFA Rules not processed for user `cd3c04f39d774998b66e6ce920925691`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:36:33.511542 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-15229053-f4b3-4ba4-8754-57935b197c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:36:33.519031 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1108/2241] 10.222.0.25 () {58 vars in 967 bytes} [Thu Jul 7 15:36:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:36:33.531669 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a97e7a-0d0e-4c56-809b-4b3f9a651a6a service swift] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:36:33.544499 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a97e7a-0d0e-4c56-809b-4b3f9a651a6a service swift] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:36:33.552154 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d80744-60d2-495f-bd34-70cb024297c0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b18033f849b240999d12c1885d2f7b91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:36:33.552829 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d80744-60d2-495f-bd34-70cb024297c0 service swift] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:33.553111 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d80744-60d2-495f-bd34-70cb024297c0 service swift] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:33.553480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d80744-60d2-495f-bd34-70cb024297c0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:33.567420 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d80744-60d2-495f-bd34-70cb024297c0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:36:33.569953 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d80744-60d2-495f-bd34-70cb024297c0 service swift] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:36:33.583459 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1134/2242] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Jul 7 15:36:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 07 15:36:34.206548 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb38ed-3410-4d5b-982e-fdbd84c145ee None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:34.206876 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb38ed-3410-4d5b-982e-fdbd84c145ee None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:34.207152 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb38ed-3410-4d5b-982e-fdbd84c145ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:34.226264 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-4fcb38ed-3410-4d5b-982e-fdbd84c145ee None None] MFA Rules not processed for user `cd3c04f39d774998b66e6ce920925691`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:36:34.227539 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-4fcb38ed-3410-4d5b-982e-fdbd84c145ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:36:34.234800 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1109/2243] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Jul 7 15:36:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:36:34.247701 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d80744-60d2-495f-bd34-70cb024297c0 service swift] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:36:34.260614 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d80744-60d2-495f-bd34-70cb024297c0 service swift] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:36:34.267215 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511455b-ca9b-43fc-80e4-3aafa6c14a7d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b18033f849b240999d12c1885d2f7b91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:36:34.271439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511455b-ca9b-43fc-80e4-3aafa6c14a7d service swift] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:34.271439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511455b-ca9b-43fc-80e4-3aafa6c14a7d service swift] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:34.271439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511455b-ca9b-43fc-80e4-3aafa6c14a7d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:34.276745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c511455b-ca9b-43fc-80e4-3aafa6c14a7d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:36:34.278360 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c511455b-ca9b-43fc-80e4-3aafa6c14a7d service swift] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:36:34.286509 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1135/2244] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Jul 7 15:36:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 07 15:36:34.835848 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af052089-5cf0-4c3d-b235-439a2a46eda6 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:34.836291 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af052089-5cf0-4c3d-b235-439a2a46eda6 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:34.836414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af052089-5cf0-4c3d-b235-439a2a46eda6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:34.853841 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-af052089-5cf0-4c3d-b235-439a2a46eda6 None None] MFA Rules not processed for user `cd3c04f39d774998b66e6ce920925691`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:36:34.855506 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-af052089-5cf0-4c3d-b235-439a2a46eda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:36:34.863300 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1110/2245] 10.222.0.25 () {58 vars in 967 bytes} [Thu Jul 7 15:36:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:36:34.875272 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511455b-ca9b-43fc-80e4-3aafa6c14a7d service swift] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:36:34.886018 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511455b-ca9b-43fc-80e4-3aafa6c14a7d service swift] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:36:34.890902 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fabb21-c8ff-45a2-8ea7-91524c56d7e1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b18033f849b240999d12c1885d2f7b91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:36:34.891528 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fabb21-c8ff-45a2-8ea7-91524c56d7e1 service swift] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:34.891838 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fabb21-c8ff-45a2-8ea7-91524c56d7e1 service swift] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:34.892233 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fabb21-c8ff-45a2-8ea7-91524c56d7e1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:34.900908 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fabb21-c8ff-45a2-8ea7-91524c56d7e1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:36:34.902405 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fabb21-c8ff-45a2-8ea7-91524c56d7e1 service swift] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:36:34.910860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1136/2246] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Jul 7 15:36:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 07 15:36:37.732621 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af052089-5cf0-4c3d-b235-439a2a46eda6 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:36:37.797125 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af052089-5cf0-4c3d-b235-439a2a46eda6 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:36:37.802396 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e4167-4c26-4e67-8266-4cc9882b906b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:36:37.803334 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e4167-4c26-4e67-8266-4cc9882b906b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:37.803647 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e4167-4c26-4e67-8266-4cc9882b906b admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:37.803922 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e4167-4c26-4e67-8266-4cc9882b906b admin admin] PATH_INFO: `/v3/users/3256d562f0a64600baeb215ebff4190d` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:37.805104 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7e4167-4c26-4e67-8266-4cc9882b906b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3256d562f0a64600baeb215ebff4190d)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:36:37.807035 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7e4167-4c26-4e67-8266-4cc9882b906b admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:36:38.027714 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-bb7e4167-4c26-4e67-8266-4cc9882b906b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3256d562f0a64600baeb215ebff4190d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:36:38.035272 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-bb7e4167-4c26-4e67-8266-4cc9882b906b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3256d562f0a64600baeb215ebff4190d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:36:38.047464 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-bb7e4167-4c26-4e67-8266-4cc9882b906b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3256d562f0a64600baeb215ebff4190d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:36:38.048150 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-bb7e4167-4c26-4e67-8266-4cc9882b906b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3256d562f0a64600baeb215ebff4190d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:36:38.104748 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-bb7e4167-4c26-4e67-8266-4cc9882b906b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3256d562f0a64600baeb215ebff4190d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:36:38.111247 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1111/2247] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:36:37 2022] DELETE /identity/v3/users/3256d562f0a64600baeb215ebff4190d => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:36:38.115722 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129e569-973b-4a13-aa5a-e38edfef1bd4 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:38.116169 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129e569-973b-4a13-aa5a-e38edfef1bd4 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:38.116262 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129e569-973b-4a13-aa5a-e38edfef1bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:38.133347 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-1129e569-973b-4a13-aa5a-e38edfef1bd4 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:36:38.134534 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-1129e569-973b-4a13-aa5a-e38edfef1bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:36:38.203208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1137/2248] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:36:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:36:38.213034 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e4167-4c26-4e67-8266-4cc9882b906b admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:36:38.214486 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-bb7e4167-4c26-4e67-8266-4cc9882b906b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:36:38.274658 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e4167-4c26-4e67-8266-4cc9882b906b admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:36:38.278208 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e456288-a9ec-420d-b5f3-14a009d41167 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:36:38.278883 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e456288-a9ec-420d-b5f3-14a009d41167 service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:38.279163 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e456288-a9ec-420d-b5f3-14a009d41167 service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:38.279437 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e456288-a9ec-420d-b5f3-14a009d41167 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:38.287001 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:36:38.289450 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e456288-a9ec-420d-b5f3-14a009d41167 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:36:38.290854 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e456288-a9ec-420d-b5f3-14a009d41167 service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:36:38.301513 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1112/2249] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:36:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:36:38.556220 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129e569-973b-4a13-aa5a-e38edfef1bd4 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:36:38.556220 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-1129e569-973b-4a13-aa5a-e38edfef1bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:36:38.568552 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129e569-973b-4a13-aa5a-e38edfef1bd4 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:36:38.573562 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6b6621-a9a1-4465-8934-f7791da54cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:36:38.574433 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6b6621-a9a1-4465-8934-f7791da54cf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:38.574716 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6b6621-a9a1-4465-8934-f7791da54cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:38.574997 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6b6621-a9a1-4465-8934-f7791da54cf2 admin admin] PATH_INFO: `/v3/projects/b42fb18bea524ba980012973ec2a9ee2` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:38.576220 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6b6621-a9a1-4465-8934-f7791da54cf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b42fb18bea524ba980012973ec2a9ee2)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:36:38.579041 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6b6621-a9a1-4465-8934-f7791da54cf2 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:36:38.706411 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-2a6b6621-a9a1-4465-8934-f7791da54cf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b42fb18bea524ba980012973ec2a9ee2', 'request_id': 'req-2a6b6621-a9a1-4465-8934-f7791da54cf2'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:36:38.715674 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-2a6b6621-a9a1-4465-8934-f7791da54cf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b42fb18bea524ba980012973ec2a9ee2', 'request_id': 'req-2a6b6621-a9a1-4465-8934-f7791da54cf2'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:36:38.722021 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-2a6b6621-a9a1-4465-8934-f7791da54cf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b42fb18bea524ba980012973ec2a9ee2', 'request_id': 'req-2a6b6621-a9a1-4465-8934-f7791da54cf2'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:36:38.821601 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-2a6b6621-a9a1-4465-8934-f7791da54cf2 admin admin] The token cache is being invalidate because project b42fb18bea524ba980012973ec2a9ee2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:36:38.821815 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-2a6b6621-a9a1-4465-8934-f7791da54cf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:36:38.822423 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-2a6b6621-a9a1-4465-8934-f7791da54cf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:36:38.824028 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1138/2250] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:36:38 2022] DELETE /identity/v3/projects/b42fb18bea524ba980012973ec2a9ee2 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:36:38.829014 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a6e80-7723-4897-ae1b-8a824d949af1 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:38.829456 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a6e80-7723-4897-ae1b-8a824d949af1 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:38.829579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a6e80-7723-4897-ae1b-8a824d949af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:38.847111 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-825a6e80-7723-4897-ae1b-8a824d949af1 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:36:38.848341 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-825a6e80-7723-4897-ae1b-8a824d949af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:36:38.911491 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1113/2251] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:36:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:36:38.919179 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b91b44-9364-40f9-9a97-82a5dba804d2 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:38.919469 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b91b44-9364-40f9-9a97-82a5dba804d2 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:38.919773 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b91b44-9364-40f9-9a97-82a5dba804d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:38.937975 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-55b91b44-9364-40f9-9a97-82a5dba804d2 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:36:38.939787 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-55b91b44-9364-40f9-9a97-82a5dba804d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:36:38.947740 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1139/2252] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:36:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:36:38.953193 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a6e80-7723-4897-ae1b-8a824d949af1 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:36:38.963277 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a6e80-7723-4897-ae1b-8a824d949af1 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:36:38.967922 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0146ff-b31f-4672-af66-cfad4b7c79d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:36:38.968578 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0146ff-b31f-4672-af66-cfad4b7c79d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:38.968952 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0146ff-b31f-4672-af66-cfad4b7c79d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:38.969233 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0146ff-b31f-4672-af66-cfad4b7c79d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:38.969968 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0146ff-b31f-4672-af66-cfad4b7c79d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:36:38.970303 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0146ff-b31f-4672-af66-cfad4b7c79d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:36:38.971921 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0146ff-b31f-4672-af66-cfad4b7c79d2 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:36:38.984148 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1114/2253] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:36:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:36:38.989017 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b91b44-9364-40f9-9a97-82a5dba804d2 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:36:38.998777 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b91b44-9364-40f9-9a97-82a5dba804d2 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:36:39.004145 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3aa65e-a15e-498e-bffa-3faf6891adf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:36:39.004788 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3aa65e-a15e-498e-bffa-3faf6891adf2 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:39.005073 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3aa65e-a15e-498e-bffa-3faf6891adf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:39.014673 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3aa65e-a15e-498e-bffa-3faf6891adf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:39.014673 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3aa65e-a15e-498e-bffa-3faf6891adf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:36:39.014673 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3aa65e-a15e-498e-bffa-3faf6891adf2 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:36:39.098235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1140/2254] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:36:38 2022] POST /identity/v3/projects => generated 379 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:36:39.102367 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0146ff-b31f-4672-af66-cfad4b7c79d2 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:36:39.113107 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0146ff-b31f-4672-af66-cfad4b7c79d2 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:36:39.167233 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcac785-bdf8-49e1-a863-ae747387bb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:36:39.167943 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcac785-bdf8-49e1-a863-ae747387bb41 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:39.168226 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcac785-bdf8-49e1-a863-ae747387bb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:39.168500 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcac785-bdf8-49e1-a863-ae747387bb41 admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:39.169174 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcac785-bdf8-49e1-a863-ae747387bb41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:36:39.171071 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcac785-bdf8-49e1-a863-ae747387bb41 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:36:39.233103 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1115/2255] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:36:39 2022] POST /identity/v3/users => generated 290 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:36:39.237129 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3aa65e-a15e-498e-bffa-3faf6891adf2 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:36:39.242788 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:36:39.248105 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3aa65e-a15e-498e-bffa-3faf6891adf2 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:36:39.253361 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a4a5fd-7abe-4c11-a2da-07f0772f5813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:36:39.254012 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4a5fd-7abe-4c11-a2da-07f0772f5813 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:39.254319 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4a5fd-7abe-4c11-a2da-07f0772f5813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:39.254635 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4a5fd-7abe-4c11-a2da-07f0772f5813 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:39.255368 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a4a5fd-7abe-4c11-a2da-07f0772f5813 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:36:39.257269 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a4a5fd-7abe-4c11-a2da-07f0772f5813 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:36:39.268336 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1141/2256] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:36:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:36:39.272180 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcac785-bdf8-49e1-a863-ae747387bb41 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:36:39.282582 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcac785-bdf8-49e1-a863-ae747387bb41 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:36:39.287218 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcadf6e-1440-460e-9788-44a84330d038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:36:39.287907 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcadf6e-1440-460e-9788-44a84330d038 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:39.288196 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcadf6e-1440-460e-9788-44a84330d038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:39.288476 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcadf6e-1440-460e-9788-44a84330d038 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:39.289269 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcadf6e-1440-460e-9788-44a84330d038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:36:39.291178 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcadf6e-1440-460e-9788-44a84330d038 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:36:39.300866 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1116/2257] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:36:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:36:39.304744 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a4a5fd-7abe-4c11-a2da-07f0772f5813 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:36:39.315580 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a4a5fd-7abe-4c11-a2da-07f0772f5813 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:36:39.320146 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c35e941-23e7-4f85-b7a2-f3b4cc374afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:36:39.325258 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35e941-23e7-4f85-b7a2-f3b4cc374afc admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:39.325258 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35e941-23e7-4f85-b7a2-f3b4cc374afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:39.325258 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35e941-23e7-4f85-b7a2-f3b4cc374afc admin admin] PATH_INFO: `/v3/projects/b72dae7a2e644460b4c63da884023c82/users/05e0f1ddeb224677b18cfc769dc45d47/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:39.336818 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c35e941-23e7-4f85-b7a2-f3b4cc374afc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b72dae7a2e644460b4c63da884023c82, user_id=05e0f1ddeb224677b18cfc769dc45d47, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:36:39.339432 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c35e941-23e7-4f85-b7a2-f3b4cc374afc admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:36:39.455622 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1142/2258] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:36:39 2022] PUT /identity/v3/projects/b72dae7a2e644460b4c63da884023c82/users/05e0f1ddeb224677b18cfc769dc45d47/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:36:39.460519 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc480fb7-c816-49e9-9e7a-e9d4a3ddb735 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:39.460815 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc480fb7-c816-49e9-9e7a-e9d4a3ddb735 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:39.461765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc480fb7-c816-49e9-9e7a-e9d4a3ddb735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:39.480994 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-fc480fb7-c816-49e9-9e7a-e9d4a3ddb735 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:36:39.482261 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-fc480fb7-c816-49e9-9e7a-e9d4a3ddb735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:36:39.542793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1117/2259] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:36:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:36:39.552423 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c35e941-23e7-4f85-b7a2-f3b4cc374afc admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:36:39.554184 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-4c35e941-23e7-4f85-b7a2-f3b4cc374afc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:36:39.611453 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c35e941-23e7-4f85-b7a2-f3b4cc374afc admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:36:39.615207 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fba4ec4-f685-4300-ae14-0d3883986d98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:36:39.615892 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba4ec4-f685-4300-ae14-0d3883986d98 service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:39.616241 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba4ec4-f685-4300-ae14-0d3883986d98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:39.616523 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba4ec4-f685-4300-ae14-0d3883986d98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:39.625666 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fba4ec4-f685-4300-ae14-0d3883986d98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:36:39.627180 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fba4ec4-f685-4300-ae14-0d3883986d98 service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:36:39.636671 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1143/2260] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:36:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 15:36:47.612944 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e240b06-464a-4e16-a7cb-6d37ff06ba46 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:47.613299 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e240b06-464a-4e16-a7cb-6d37ff06ba46 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:47.613299 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e240b06-464a-4e16-a7cb-6d37ff06ba46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:47.620565 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:36:47.637208 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-3e240b06-464a-4e16-a7cb-6d37ff06ba46 None None] MFA Rules not processed for user `cd3c04f39d774998b66e6ce920925691`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:36:47.638583 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-3e240b06-464a-4e16-a7cb-6d37ff06ba46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:36:47.715095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1118/2261] 10.222.0.25 () {58 vars in 967 bytes} [Thu Jul 7 15:36:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:36:47.742811 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fba4ec4-f685-4300-ae14-0d3883986d98 service neutron] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:36:47.744343 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-0fba4ec4-f685-4300-ae14-0d3883986d98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:36:47.827858 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fba4ec4-f685-4300-ae14-0d3883986d98 service neutron] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:36:47.834729 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e6c40-9559-4bc1-9d9c-bbb0ecfe1428 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b18033f849b240999d12c1885d2f7b91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:36:47.835746 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e6c40-9559-4bc1-9d9c-bbb0ecfe1428 service swift] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:47.836248 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e6c40-9559-4bc1-9d9c-bbb0ecfe1428 service swift] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:47.836722 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e6c40-9559-4bc1-9d9c-bbb0ecfe1428 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:47.848617 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510e6c40-9559-4bc1-9d9c-bbb0ecfe1428 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:36:47.850532 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510e6c40-9559-4bc1-9d9c-bbb0ecfe1428 service swift] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:36:47.861490 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1144/2262] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Jul 7 15:36:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 07 15:36:47.992927 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04895e1-1e9a-4877-a538-cb99c8b0801c None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:47.992927 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04895e1-1e9a-4877-a538-cb99c8b0801c None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:47.993144 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04895e1-1e9a-4877-a538-cb99c8b0801c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:48.082394 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-d04895e1-1e9a-4877-a538-cb99c8b0801c None None] MFA Rules not processed for user `05e0f1ddeb224677b18cfc769dc45d47`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:36:48.089718 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-d04895e1-1e9a-4877-a538-cb99c8b0801c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:36:48.228552 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1119/2263] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:36:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:36:48.229572 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e6c40-9559-4bc1-9d9c-bbb0ecfe1428 service swift] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:36:48.250133 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e6c40-9559-4bc1-9d9c-bbb0ecfe1428 service swift] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:36:48.258151 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e7682-280a-46ec-b2a0-b80c545c2888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:36:48.259651 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e7682-280a-46ec-b2a0-b80c545c2888 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:48.260058 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e7682-280a-46ec-b2a0-b80c545c2888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:48.260343 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e7682-280a-46ec-b2a0-b80c545c2888 admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:48.261057 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5e7682-280a-46ec-b2a0-b80c545c2888 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:36:48.263190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5e7682-280a-46ec-b2a0-b80c545c2888 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:36:48.486434 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1145/2264] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:36:48 2022] POST /identity/v3/projects => generated 379 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:36:48.490451 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04895e1-1e9a-4877-a538-cb99c8b0801c None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:36:48.505217 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04895e1-1e9a-4877-a538-cb99c8b0801c None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:36:48.650683 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e97dd54-46b1-4c7a-b512-564d33ac81e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:36:48.651476 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97dd54-46b1-4c7a-b512-564d33ac81e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:48.655744 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97dd54-46b1-4c7a-b512-564d33ac81e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:48.655744 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97dd54-46b1-4c7a-b512-564d33ac81e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:48.655744 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e97dd54-46b1-4c7a-b512-564d33ac81e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:36:48.657134 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e97dd54-46b1-4c7a-b512-564d33ac81e8 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:36:48.819117 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1120/2265] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:36:48 2022] POST /identity/v3/users => generated 296 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:36:48.823794 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e7682-280a-46ec-b2a0-b80c545c2888 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:36:48.843672 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e7682-280a-46ec-b2a0-b80c545c2888 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:36:48.851601 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccebbaa-eab3-4a1a-8529-6283f7ff292b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:36:48.852468 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccebbaa-eab3-4a1a-8529-6283f7ff292b admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:48.852777 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccebbaa-eab3-4a1a-8529-6283f7ff292b admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:48.853223 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccebbaa-eab3-4a1a-8529-6283f7ff292b admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:48.854111 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccebbaa-eab3-4a1a-8529-6283f7ff292b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:36:48.859869 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccebbaa-eab3-4a1a-8529-6283f7ff292b admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:36:48.896192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e97dd54-46b1-4c7a-b512-564d33ac81e8 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:36:48.896641 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1146/2266] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:36:48.904192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e97dd54-46b1-4c7a-b512-564d33ac81e8 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:36:48.910224 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15063556-7e0c-4198-b266-e44977eb856b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:36:48.911220 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15063556-7e0c-4198-b266-e44977eb856b admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:48.911681 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15063556-7e0c-4198-b266-e44977eb856b admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:48.913392 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15063556-7e0c-4198-b266-e44977eb856b admin admin] PATH_INFO: `/v3/projects/98953ecda1544abdb7652065dafa75b3/users/840981beb61a41e79b70ee6ffff3e511/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:48.930053 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15063556-7e0c-4198-b266-e44977eb856b admin admin] RBAC: Authorizing `identity:create_grant(project_id=98953ecda1544abdb7652065dafa75b3, user_id=840981beb61a41e79b70ee6ffff3e511, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:36:48.934172 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15063556-7e0c-4198-b266-e44977eb856b admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:36:49.090767 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1121/2267] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:36:48 2022] PUT /identity/v3/projects/98953ecda1544abdb7652065dafa75b3/users/840981beb61a41e79b70ee6ffff3e511/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:36:49.097067 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccebbaa-eab3-4a1a-8529-6283f7ff292b admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:36:49.121367 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccebbaa-eab3-4a1a-8529-6283f7ff292b admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:36:49.161898 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:36:49.195764 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de9eb8-b594-4816-acab-7a06ca63471b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:36:49.196951 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de9eb8-b594-4816-acab-7a06ca63471b admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:49.197534 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de9eb8-b594-4816-acab-7a06ca63471b admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:49.197991 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de9eb8-b594-4816-acab-7a06ca63471b admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:49.198951 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42de9eb8-b594-4816-acab-7a06ca63471b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:36:49.200786 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42de9eb8-b594-4816-acab-7a06ca63471b admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:36:49.241528 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15063556-7e0c-4198-b266-e44977eb856b admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:36:49.242238 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1147/2268] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:36:49.248971 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:36:49.264605 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15063556-7e0c-4198-b266-e44977eb856b admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:36:49.270378 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f153b5-8e99-470e-b5c1-9a3860ace3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:36:49.270822 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f153b5-8e99-470e-b5c1-9a3860ace3fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:49.271339 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f153b5-8e99-470e-b5c1-9a3860ace3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:49.271690 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f153b5-8e99-470e-b5c1-9a3860ace3fb admin admin] PATH_INFO: `/v3/projects/98953ecda1544abdb7652065dafa75b3/users/840981beb61a41e79b70ee6ffff3e511/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:49.273815 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f153b5-8e99-470e-b5c1-9a3860ace3fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=98953ecda1544abdb7652065dafa75b3, user_id=840981beb61a41e79b70ee6ffff3e511, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:36:49.276779 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f153b5-8e99-470e-b5c1-9a3860ace3fb admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:36:49.293491 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1122/2269] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:36:49 2022] PUT /identity/v3/projects/98953ecda1544abdb7652065dafa75b3/users/840981beb61a41e79b70ee6ffff3e511/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:36:55.536882 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d77280-262b-4803-83f5-c4306eccc290 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:55.537432 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d77280-262b-4803-83f5-c4306eccc290 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:55.537432 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d77280-262b-4803-83f5-c4306eccc290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:55.569066 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-58d77280-262b-4803-83f5-c4306eccc290 None None] MFA Rules not processed for user `840981beb61a41e79b70ee6ffff3e511`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:36:55.570306 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-58d77280-262b-4803-83f5-c4306eccc290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:36:55.648023 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1148/2270] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:36:55 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:36:55.664844 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f153b5-8e99-470e-b5c1-9a3860ace3fb admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:36:55.671280 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-d5f153b5-8e99-470e-b5c1-9a3860ace3fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:36:55.779104 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f153b5-8e99-470e-b5c1-9a3860ace3fb admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:36:55.788113 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a86c7ef-18cf-4614-81e6-8e817e983568 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:36:55.789119 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86c7ef-18cf-4614-81e6-8e817e983568 service nova] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:36:55.789590 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86c7ef-18cf-4614-81e6-8e817e983568 service nova] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:36:55.790060 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86c7ef-18cf-4614-81e6-8e817e983568 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:36:55.857908 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a86c7ef-18cf-4614-81e6-8e817e983568 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:36:55.859076 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a86c7ef-18cf-4614-81e6-8e817e983568 service nova] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:36:55.871596 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1123/2271] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:36:55 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:37:01.807185 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d77280-262b-4803-83f5-c4306eccc290 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:37:01.808816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-58d77280-262b-4803-83f5-c4306eccc290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:37:01.865400 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d77280-262b-4803-83f5-c4306eccc290 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:37:01.869069 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d595a363-cfa4-4da7-ac21-c1e3239f89b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:37:01.869703 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595a363-cfa4-4da7-ac21-c1e3239f89b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:37:01.869981 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595a363-cfa4-4da7-ac21-c1e3239f89b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:37:01.870255 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595a363-cfa4-4da7-ac21-c1e3239f89b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:37:01.872063 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-d595a363-cfa4-4da7-ac21-c1e3239f89b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:37:01.938094 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d595a363-cfa4-4da7-ac21-c1e3239f89b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:37:01.940296 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d595a363-cfa4-4da7-ac21-c1e3239f89b8 service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:37:01.944291 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:37:01.952651 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1149/2272] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:37:01 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:37:04.308873 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49d813-586b-495c-be2c-cfb70bf9032d None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:37:04.309404 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49d813-586b-495c-be2c-cfb70bf9032d None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:37:04.309593 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49d813-586b-495c-be2c-cfb70bf9032d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:37:04.328288 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-8c49d813-586b-495c-be2c-cfb70bf9032d None None] MFA Rules not processed for user `b2e09539caff4d4698cf29c1ea22d365`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:37:04.329646 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-8c49d813-586b-495c-be2c-cfb70bf9032d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:37:04.403045 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1124/2273] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:37:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:37:11.976073 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:37:14.392870 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=85,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:37:17.035801 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc84fd-2ece-47c0-8ed6-91ace0caa460 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:37:17.036364 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc84fd-2ece-47c0-8ed6-91ace0caa460 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:37:17.036364 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc84fd-2ece-47c0-8ed6-91ace0caa460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:37:17.052813 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-3cdc84fd-2ece-47c0-8ed6-91ace0caa460 None None] MFA Rules not processed for user `05e0f1ddeb224677b18cfc769dc45d47`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:37:17.054161 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-3cdc84fd-2ece-47c0-8ed6-91ace0caa460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:37:17.061000 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1150/2274] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:37:17 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 26 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:37:27.048129 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=3 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:37:36.014980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c49d813-586b-495c-be2c-cfb70bf9032d None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:37:36.024441 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c49d813-586b-495c-be2c-cfb70bf9032d None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:37:36.027994 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0659fa-0f67-49f9-85f4-36980abbaafb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:37:36.028626 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0659fa-0f67-49f9-85f4-36980abbaafb service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:37:36.028906 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0659fa-0f67-49f9-85f4-36980abbaafb service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:37:36.029180 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0659fa-0f67-49f9-85f4-36980abbaafb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:37:36.038847 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0659fa-0f67-49f9-85f4-36980abbaafb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:37:36.040395 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0659fa-0f67-49f9-85f4-36980abbaafb service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:37:36.050339 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1125/2275] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:37:36 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:37:46.045476 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=6 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:37:52.188454 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5750ba9f-11b0-41bd-9b62-d020ee51433d None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:37:52.192460 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5750ba9f-11b0-41bd-9b62-d020ee51433d None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:37:52.192460 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5750ba9f-11b0-41bd-9b62-d020ee51433d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:37:52.207245 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-5750ba9f-11b0-41bd-9b62-d020ee51433d None None] MFA Rules not processed for user `2033ad40d45747aa8294e4feb5a6cfe0`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:37:52.208471 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-5750ba9f-11b0-41bd-9b62-d020ee51433d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:37:52.263532 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1151/2276] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:37:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:37:58.971746 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0659fa-0f67-49f9-85f4-36980abbaafb service neutron] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:37:58.980910 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0659fa-0f67-49f9-85f4-36980abbaafb service neutron] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:37:58.985351 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8818e0ee-e9e8-4553-9f4b-d6441e78ddbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:37:58.986813 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818e0ee-e9e8-4553-9f4b-d6441e78ddbd service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:37:58.987161 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818e0ee-e9e8-4553-9f4b-d6441e78ddbd service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:37:58.987537 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818e0ee-e9e8-4553-9f4b-d6441e78ddbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:37:58.990083 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-8818e0ee-e9e8-4553-9f4b-d6441e78ddbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:37:59.007367 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8818e0ee-e9e8-4553-9f4b-d6441e78ddbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:37:59.009588 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8818e0ee-e9e8-4553-9f4b-d6441e78ddbd service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:37:59.023401 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1126/2277] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:37:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:38:01.422370 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5750ba9f-11b0-41bd-9b62-d020ee51433d None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:38:01.432906 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5750ba9f-11b0-41bd-9b62-d020ee51433d None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:38:01.436987 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1342c5d-7e49-439b-891b-ec332a143354 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:38:01.447727 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1342c5d-7e49-439b-891b-ec332a143354 service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:38:01.447727 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1342c5d-7e49-439b-891b-ec332a143354 service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:38:01.447727 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1342c5d-7e49-439b-891b-ec332a143354 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:38:01.447727 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-b1342c5d-7e49-439b-891b-ec332a143354 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:38:01.454390 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1342c5d-7e49-439b-891b-ec332a143354 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:38:01.455895 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1342c5d-7e49-439b-891b-ec332a143354 service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:38:01.467436 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1152/2278] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:38:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:38:02.893329 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f97526-db2d-419f-80ca-3601d300a63c None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:38:02.893826 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f97526-db2d-419f-80ca-3601d300a63c None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:38:02.893924 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f97526-db2d-419f-80ca-3601d300a63c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:38:02.911032 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-57f97526-db2d-419f-80ca-3601d300a63c None None] MFA Rules not processed for user `9528a9a3306d49b0978d77d1ce1ca900`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:38:02.912194 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-57f97526-db2d-419f-80ca-3601d300a63c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:38:02.967137 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1127/2279] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:38:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:38:11.462034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=31 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:38:12.960394 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=31 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:38:17.050149 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1342c5d-7e49-439b-891b-ec332a143354 service neutron] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:38:17.051694 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-b1342c5d-7e49-439b-891b-ec332a143354 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:38:17.129604 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1342c5d-7e49-439b-891b-ec332a143354 service neutron] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:38:17.133379 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646663e6-e497-4656-8045-01601d4a5e21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:38:17.134069 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646663e6-e497-4656-8045-01601d4a5e21 service cinder] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:38:17.134300 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646663e6-e497-4656-8045-01601d4a5e21 service cinder] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:38:17.134576 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646663e6-e497-4656-8045-01601d4a5e21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:38:17.136395 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-646663e6-e497-4656-8045-01601d4a5e21 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:38:17.198576 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646663e6-e497-4656-8045-01601d4a5e21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:38:17.200763 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646663e6-e497-4656-8045-01601d4a5e21 service cinder] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:38:17.210204 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1153/2280] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:38:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:38:17.262981 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f97526-db2d-419f-80ca-3601d300a63c None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:38:17.264464 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-57f97526-db2d-419f-80ca-3601d300a63c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:38:17.276525 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f97526-db2d-419f-80ca-3601d300a63c None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:38:17.280107 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826ae2f-6c36-49cb-b6e9-eab4d9d6dacf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:38:17.280750 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826ae2f-6c36-49cb-b6e9-eab4d9d6dacf service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:38:17.281034 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826ae2f-6c36-49cb-b6e9-eab4d9d6dacf service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:38:17.281310 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826ae2f-6c36-49cb-b6e9-eab4d9d6dacf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:38:17.283653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-2826ae2f-6c36-49cb-b6e9-eab4d9d6dacf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:38:17.294952 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2826ae2f-6c36-49cb-b6e9-eab4d9d6dacf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:38:17.296419 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2826ae2f-6c36-49cb-b6e9-eab4d9d6dacf service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:38:17.306964 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1128/2281] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:38:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:38:27.205718 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=57 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:38:27.302693 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=6 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:38:27.593173 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1492fe-7c8d-4b86-a6c0-c3541c8bd1ae None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:38:27.593463 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1492fe-7c8d-4b86-a6c0-c3541c8bd1ae None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:38:27.593737 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1492fe-7c8d-4b86-a6c0-c3541c8bd1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:38:27.611765 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-cd1492fe-7c8d-4b86-a6c0-c3541c8bd1ae None None] MFA Rules not processed for user `b5dec80a83aa463b99bd298ab09dfbdb`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:38:27.612979 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-cd1492fe-7c8d-4b86-a6c0-c3541c8bd1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:38:27.619908 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1154/2282] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:38:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:38:27.973261 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826ae2f-6c36-49cb-b6e9-eab4d9d6dacf service cinder] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:38:27.982539 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826ae2f-6c36-49cb-b6e9-eab4d9d6dacf service cinder] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:38:27.986437 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863bca36-4293-48b9-992a-d1cf1f406b7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:38:27.987085 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863bca36-4293-48b9-992a-d1cf1f406b7c service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:38:27.987351 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863bca36-4293-48b9-992a-d1cf1f406b7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:38:27.987653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863bca36-4293-48b9-992a-d1cf1f406b7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:38:27.989421 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-863bca36-4293-48b9-992a-d1cf1f406b7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:38:27.999198 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863bca36-4293-48b9-992a-d1cf1f406b7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:38:28.000668 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863bca36-4293-48b9-992a-d1cf1f406b7c service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:38:28.009439 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1129/2283] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:38:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:38:28.385771 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1492fe-7c8d-4b86-a6c0-c3541c8bd1ae None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:38:28.387792 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-cd1492fe-7c8d-4b86-a6c0-c3541c8bd1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:38:28.463694 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1492fe-7c8d-4b86-a6c0-c3541c8bd1ae None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:38:28.468674 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a67230-523e-4ba3-9281-bec4b64d4657 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '755c2782cfb042f89950b8fc862e4fa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:38:28.471901 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a67230-523e-4ba3-9281-bec4b64d4657 service placement] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:38:28.471901 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a67230-523e-4ba3-9281-bec4b64d4657 service placement] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:38:28.471901 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a67230-523e-4ba3-9281-bec4b64d4657 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:38:28.471901 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-d1a67230-523e-4ba3-9281-bec4b64d4657 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:38:28.482999 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a67230-523e-4ba3-9281-bec4b64d4657 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:38:28.484980 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a67230-523e-4ba3-9281-bec4b64d4657 service placement] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:38:28.495318 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1155/2284] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:38:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:38:30.016284 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae09b4-c562-46a9-a0ac-56c9d5451207 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:38:30.016803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae09b4-c562-46a9-a0ac-56c9d5451207 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:38:30.016909 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae09b4-c562-46a9-a0ac-56c9d5451207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:38:30.035484 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-97ae09b4-c562-46a9-a0ac-56c9d5451207 None None] MFA Rules not processed for user `b2e09539caff4d4698cf29c1ea22d365`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:38:30.037756 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-97ae09b4-c562-46a9-a0ac-56c9d5451207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:38:30.044269 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1130/2285] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:38:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:38:30.347817 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a67230-523e-4ba3-9281-bec4b64d4657 service placement] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:38:30.356951 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a67230-523e-4ba3-9281-bec4b64d4657 service placement] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:38:30.361145 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4bf018-00ec-4e6e-ae8f-c79ee7c9b649 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:38:30.362339 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4bf018-00ec-4e6e-ae8f-c79ee7c9b649 service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:38:30.362753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4bf018-00ec-4e6e-ae8f-c79ee7c9b649 service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:38:30.363034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4bf018-00ec-4e6e-ae8f-c79ee7c9b649 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:38:30.364827 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-da4bf018-00ec-4e6e-ae8f-c79ee7c9b649 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:38:30.374527 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4bf018-00ec-4e6e-ae8f-c79ee7c9b649 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:38:30.375971 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4bf018-00ec-4e6e-ae8f-c79ee7c9b649 service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:38:30.384513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1156/2286] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:38:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:38:31.610703 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae09b4-c562-46a9-a0ac-56c9d5451207 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:38:31.620284 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae09b4-c562-46a9-a0ac-56c9d5451207 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:38:31.623854 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c872a88-6def-4364-a2cd-37bdf931b13c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:38:31.624486 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c872a88-6def-4364-a2cd-37bdf931b13c service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:38:31.624803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c872a88-6def-4364-a2cd-37bdf931b13c service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:38:31.627761 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c872a88-6def-4364-a2cd-37bdf931b13c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:38:31.627761 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-8c872a88-6def-4364-a2cd-37bdf931b13c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:38:31.636876 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c872a88-6def-4364-a2cd-37bdf931b13c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:38:31.638372 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c872a88-6def-4364-a2cd-37bdf931b13c service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:38:31.646868 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1131/2287] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:38:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:38:35.688524 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4bf018-00ec-4e6e-ae8f-c79ee7c9b649 service neutron] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:38:35.692260 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-da4bf018-00ec-4e6e-ae8f-c79ee7c9b649 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:38:35.717188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4bf018-00ec-4e6e-ae8f-c79ee7c9b649 service neutron] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:38:35.724279 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a7e7da-b8af-4d6c-b166-4e96a0f42756 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:38:35.728429 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7e7da-b8af-4d6c-b166-4e96a0f42756 service nova] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:38:35.728429 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7e7da-b8af-4d6c-b166-4e96a0f42756 service nova] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:38:35.728429 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7e7da-b8af-4d6c-b166-4e96a0f42756 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:38:35.728429 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-c4a7e7da-b8af-4d6c-b166-4e96a0f42756 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:38:35.743502 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a7e7da-b8af-4d6c-b166-4e96a0f42756 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:38:35.745135 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a7e7da-b8af-4d6c-b166-4e96a0f42756 service nova] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:38:35.767606 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1157/2288] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:38:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:38:41.643242 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=15 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:38:45.196213 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c872a88-6def-4364-a2cd-37bdf931b13c service neutron] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:38:45.211156 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c872a88-6def-4364-a2cd-37bdf931b13c service neutron] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:38:45.217255 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6199b9-c415-48cf-bb17-e710592f88f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:38:45.223144 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6199b9-c415-48cf-bb17-e710592f88f8 service nova] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:38:45.223144 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6199b9-c415-48cf-bb17-e710592f88f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:38:45.223144 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6199b9-c415-48cf-bb17-e710592f88f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:38:45.223144 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-1c6199b9-c415-48cf-bb17-e710592f88f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:38:45.233100 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6199b9-c415-48cf-bb17-e710592f88f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:38:45.234952 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6199b9-c415-48cf-bb17-e710592f88f8 service nova] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:38:45.246700 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1132/2289] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:38:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:38:45.754061 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=47 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:38:49.873657 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a7e7da-b8af-4d6c-b166-4e96a0f42756 service nova] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:38:49.885570 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a7e7da-b8af-4d6c-b166-4e96a0f42756 service nova] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:38:49.890916 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea38a56-d4b7-4abe-bfe3-79b98f24572f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:38:49.891545 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea38a56-d4b7-4abe-bfe3-79b98f24572f service nova] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:38:49.891854 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea38a56-d4b7-4abe-bfe3-79b98f24572f service nova] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:38:49.892131 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea38a56-d4b7-4abe-bfe3-79b98f24572f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:38:49.893875 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-cea38a56-d4b7-4abe-bfe3-79b98f24572f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:38:49.904038 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea38a56-d4b7-4abe-bfe3-79b98f24572f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:38:49.905591 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea38a56-d4b7-4abe-bfe3-79b98f24572f service nova] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:38:49.914571 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1158/2290] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:38:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:38:55.242077 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=6 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:38:59.910340 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=6 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:39:12.029319 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3a64c-4004-419c-a7b0-be1107e41abb None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:12.029861 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3a64c-4004-419c-a7b0-be1107e41abb None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:12.029861 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3a64c-4004-419c-a7b0-be1107e41abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:12.048659 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-91f3a64c-4004-419c-a7b0-be1107e41abb None None] MFA Rules not processed for user `cd3c04f39d774998b66e6ce920925691`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:39:12.050365 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-91f3a64c-4004-419c-a7b0-be1107e41abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:12.117575 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1133/2291] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Jul 7 15:39:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:39:12.267880 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea38a56-d4b7-4abe-bfe3-79b98f24572f service nova] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:12.337514 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea38a56-d4b7-4abe-bfe3-79b98f24572f service nova] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:12.342811 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e86ca3c-87fe-4dc5-b107-8f6f677f100d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b18033f849b240999d12c1885d2f7b91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:12.343540 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86ca3c-87fe-4dc5-b107-8f6f677f100d service swift] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:12.343916 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86ca3c-87fe-4dc5-b107-8f6f677f100d service swift] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:12.344197 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86ca3c-87fe-4dc5-b107-8f6f677f100d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:12.353053 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e86ca3c-87fe-4dc5-b107-8f6f677f100d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:12.356685 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e86ca3c-87fe-4dc5-b107-8f6f677f100d service swift] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:12.363091 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1159/2292] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Jul 7 15:39:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 07 15:39:12.697268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f3a64c-4004-419c-a7b0-be1107e41abb None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:12.711471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f3a64c-4004-419c-a7b0-be1107e41abb None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:12.718018 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40739bef-4d37-4bd4-abf2-4f98270f9e1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:12.718958 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40739bef-4d37-4bd4-abf2-4f98270f9e1f service nova] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:12.719421 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40739bef-4d37-4bd4-abf2-4f98270f9e1f service nova] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:12.719879 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40739bef-4d37-4bd4-abf2-4f98270f9e1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:12.724838 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-40739bef-4d37-4bd4-abf2-4f98270f9e1f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:12.744507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40739bef-4d37-4bd4-abf2-4f98270f9e1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:12.746715 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40739bef-4d37-4bd4-abf2-4f98270f9e1f service nova] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:12.779944 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1134/2293] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:39:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:39:14.483407 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e86ca3c-87fe-4dc5-b107-8f6f677f100d service swift] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:14.499220 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e86ca3c-87fe-4dc5-b107-8f6f677f100d service swift] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:14.507112 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550d1cc-f101-4cdf-9632-7811f00b9e8f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '755c2782cfb042f89950b8fc862e4fa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:14.507112 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550d1cc-f101-4cdf-9632-7811f00b9e8f service placement] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:14.507112 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550d1cc-f101-4cdf-9632-7811f00b9e8f service placement] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:14.507451 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550d1cc-f101-4cdf-9632-7811f00b9e8f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:14.512130 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-f550d1cc-f101-4cdf-9632-7811f00b9e8f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:14.526998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f550d1cc-f101-4cdf-9632-7811f00b9e8f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:14.528714 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f550d1cc-f101-4cdf-9632-7811f00b9e8f service placement] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:14.553139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1160/2294] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:39:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:39:14.635093 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40739bef-4d37-4bd4-abf2-4f98270f9e1f service nova] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:14.636743 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-40739bef-4d37-4bd4-abf2-4f98270f9e1f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:14.725432 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40739bef-4d37-4bd4-abf2-4f98270f9e1f service nova] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:14.731304 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81c209-aa07-4c23-b0b0-735818cc322d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:14.732255 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81c209-aa07-4c23-b0b0-735818cc322d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:14.732536 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81c209-aa07-4c23-b0b0-735818cc322d admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:14.732810 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81c209-aa07-4c23-b0b0-735818cc322d admin admin] PATH_INFO: `/v3/users/9528a9a3306d49b0978d77d1ce1ca900` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:14.734068 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab81c209-aa07-4c23-b0b0-735818cc322d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9528a9a3306d49b0978d77d1ce1ca900)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:14.736303 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab81c209-aa07-4c23-b0b0-735818cc322d admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:14.976251 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-ab81c209-aa07-4c23-b0b0-735818cc322d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9528a9a3306d49b0978d77d1ce1ca900'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:14.984685 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-ab81c209-aa07-4c23-b0b0-735818cc322d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9528a9a3306d49b0978d77d1ce1ca900'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:15.002977 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-ab81c209-aa07-4c23-b0b0-735818cc322d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9528a9a3306d49b0978d77d1ce1ca900'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:15.003879 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-ab81c209-aa07-4c23-b0b0-735818cc322d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9528a9a3306d49b0978d77d1ce1ca900'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:15.067682 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-ab81c209-aa07-4c23-b0b0-735818cc322d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9528a9a3306d49b0978d77d1ce1ca900'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:15.075476 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1135/2295] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:39:14 2022] DELETE /identity/v3/users/9528a9a3306d49b0978d77d1ce1ca900 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:39:15.371244 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550d1cc-f101-4cdf-9632-7811f00b9e8f service placement] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:15.384500 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550d1cc-f101-4cdf-9632-7811f00b9e8f service placement] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:15.474598 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84967dd0-59b1-4e6f-9fe6-1db55db1847c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:15.475498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84967dd0-59b1-4e6f-9fe6-1db55db1847c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:15.475819 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84967dd0-59b1-4e6f-9fe6-1db55db1847c admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:15.476095 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84967dd0-59b1-4e6f-9fe6-1db55db1847c admin admin] PATH_INFO: `/v3/projects/8404651e9fb4443a9cdebb3f79a4e6a8` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:15.477415 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84967dd0-59b1-4e6f-9fe6-1db55db1847c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8404651e9fb4443a9cdebb3f79a4e6a8)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:15.479443 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84967dd0-59b1-4e6f-9fe6-1db55db1847c admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:15.582110 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-84967dd0-59b1-4e6f-9fe6-1db55db1847c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8404651e9fb4443a9cdebb3f79a4e6a8', 'request_id': 'req-84967dd0-59b1-4e6f-9fe6-1db55db1847c'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:15.589985 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-84967dd0-59b1-4e6f-9fe6-1db55db1847c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8404651e9fb4443a9cdebb3f79a4e6a8', 'request_id': 'req-84967dd0-59b1-4e6f-9fe6-1db55db1847c'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:15.597678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-84967dd0-59b1-4e6f-9fe6-1db55db1847c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8404651e9fb4443a9cdebb3f79a4e6a8', 'request_id': 'req-84967dd0-59b1-4e6f-9fe6-1db55db1847c'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:15.671469 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-84967dd0-59b1-4e6f-9fe6-1db55db1847c admin admin] The token cache is being invalidate because project 8404651e9fb4443a9cdebb3f79a4e6a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:39:15.671796 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-84967dd0-59b1-4e6f-9fe6-1db55db1847c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:15.672404 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-84967dd0-59b1-4e6f-9fe6-1db55db1847c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:15.674051 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1161/2296] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:39:15 2022] DELETE /identity/v3/projects/8404651e9fb4443a9cdebb3f79a4e6a8 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:39:15.679033 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f566f26-7bd9-4358-85f8-e97456a71e1e None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:15.679529 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f566f26-7bd9-4358-85f8-e97456a71e1e None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:15.679856 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f566f26-7bd9-4358-85f8-e97456a71e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:15.702297 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-7f566f26-7bd9-4358-85f8-e97456a71e1e None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:39:15.703872 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-7f566f26-7bd9-4358-85f8-e97456a71e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:15.858679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:39:15.858679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1136/2297] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:39:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:39:15.865314 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d85b48-8749-4f8d-9ac1-1491290bb114 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:15.865660 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d85b48-8749-4f8d-9ac1-1491290bb114 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:15.865944 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d85b48-8749-4f8d-9ac1-1491290bb114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:15.880026 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:39:15.886041 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-f2d85b48-8749-4f8d-9ac1-1491290bb114 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:39:15.887233 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-f2d85b48-8749-4f8d-9ac1-1491290bb114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:15.898573 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1162/2298] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:39:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:39:15.902958 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f566f26-7bd9-4358-85f8-e97456a71e1e None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:15.913389 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f566f26-7bd9-4358-85f8-e97456a71e1e None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:15.918621 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9eb8a9-db60-4581-a856-e8c040081073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:15.919361 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9eb8a9-db60-4581-a856-e8c040081073 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:15.919713 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9eb8a9-db60-4581-a856-e8c040081073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:15.920024 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9eb8a9-db60-4581-a856-e8c040081073 admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:15.920793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9eb8a9-db60-4581-a856-e8c040081073 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:39:15.921136 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9eb8a9-db60-4581-a856-e8c040081073 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:15.923005 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9eb8a9-db60-4581-a856-e8c040081073 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:15.936263 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1137/2299] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:39:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:39:15.941065 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d85b48-8749-4f8d-9ac1-1491290bb114 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:15.951117 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d85b48-8749-4f8d-9ac1-1491290bb114 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:15.958416 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a21b6-f622-45e3-8004-4fc029aaef0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:15.959265 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a21b6-f622-45e3-8004-4fc029aaef0c admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:15.959687 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a21b6-f622-45e3-8004-4fc029aaef0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:15.963681 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a21b6-f622-45e3-8004-4fc029aaef0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:15.964609 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416a21b6-f622-45e3-8004-4fc029aaef0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:15.966723 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416a21b6-f622-45e3-8004-4fc029aaef0c admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:16.239738 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1163/2300] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:39:15 2022] POST /identity/v3/projects => generated 373 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:39:16.243803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9eb8a9-db60-4581-a856-e8c040081073 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:16.254826 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9eb8a9-db60-4581-a856-e8c040081073 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:16.318000 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e801a5-f7cb-46f3-bef2-64018b0b548b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:16.318685 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e801a5-f7cb-46f3-bef2-64018b0b548b admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:16.318970 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e801a5-f7cb-46f3-bef2-64018b0b548b admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:16.319246 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e801a5-f7cb-46f3-bef2-64018b0b548b admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:16.320001 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e801a5-f7cb-46f3-bef2-64018b0b548b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:16.326423 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e801a5-f7cb-46f3-bef2-64018b0b548b admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:16.451505 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1138/2301] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:39:16 2022] POST /identity/v3/users => generated 287 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:39:16.456691 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a21b6-f622-45e3-8004-4fc029aaef0c admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:16.468086 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a21b6-f622-45e3-8004-4fc029aaef0c admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:16.474223 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0b3e1-5372-4239-9594-500700d32c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:16.474957 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0b3e1-5372-4239-9594-500700d32c71 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:16.475677 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0b3e1-5372-4239-9594-500700d32c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:16.475917 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0b3e1-5372-4239-9594-500700d32c71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:16.476819 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af0b3e1-5372-4239-9594-500700d32c71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:16.480629 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af0b3e1-5372-4239-9594-500700d32c71 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:16.496762 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1164/2302] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:39:16.500342 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e801a5-f7cb-46f3-bef2-64018b0b548b admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:16.514793 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e801a5-f7cb-46f3-bef2-64018b0b548b admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:16.521870 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f40e557-1a15-4135-9fac-fc630d68caed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:16.523634 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40e557-1a15-4135-9fac-fc630d68caed admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:16.524187 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40e557-1a15-4135-9fac-fc630d68caed admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:16.524789 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40e557-1a15-4135-9fac-fc630d68caed admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:16.527220 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f40e557-1a15-4135-9fac-fc630d68caed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:16.529208 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f40e557-1a15-4135-9fac-fc630d68caed admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:16.544335 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1139/2303] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:39:16.548228 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0b3e1-5372-4239-9594-500700d32c71 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:16.562722 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0b3e1-5372-4239-9594-500700d32c71 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:16.568452 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac0e28-5b36-4bb3-a4d2-124fb9ebef20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:16.569606 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac0e28-5b36-4bb3-a4d2-124fb9ebef20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:16.569992 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac0e28-5b36-4bb3-a4d2-124fb9ebef20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:16.570360 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac0e28-5b36-4bb3-a4d2-124fb9ebef20 admin admin] PATH_INFO: `/v3/projects/faef7e30953a41508d6f854702d13b0d/users/23a33ea5c6014c7eaa9f766553245d50/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:16.589265 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ac0e28-5b36-4bb3-a4d2-124fb9ebef20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=faef7e30953a41508d6f854702d13b0d, user_id=23a33ea5c6014c7eaa9f766553245d50, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:16.593780 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ac0e28-5b36-4bb3-a4d2-124fb9ebef20 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:16.666201 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1165/2304] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:39:16 2022] PUT /identity/v3/projects/faef7e30953a41508d6f854702d13b0d/users/23a33ea5c6014c7eaa9f766553245d50/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:39:16.684586 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0e0fb-b0fe-47cc-877e-f2eac6b0cb41 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:16.685121 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0e0fb-b0fe-47cc-877e-f2eac6b0cb41 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:16.685121 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0e0fb-b0fe-47cc-877e-f2eac6b0cb41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:16.716510 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-b4b0e0fb-b0fe-47cc-877e-f2eac6b0cb41 None None] MFA Rules not processed for user `23a33ea5c6014c7eaa9f766553245d50`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:39:16.716510 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-b4b0e0fb-b0fe-47cc-877e-f2eac6b0cb41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:16.794813 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1140/2305] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:39:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:39:16.811109 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac0e28-5b36-4bb3-a4d2-124fb9ebef20 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:16.813172 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-f2ac0e28-5b36-4bb3-a4d2-124fb9ebef20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:16.913377 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac0e28-5b36-4bb3-a4d2-124fb9ebef20 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:16.921675 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f621c5-1c86-4c15-9f69-ea152a2bd553 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:16.924311 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f621c5-1c86-4c15-9f69-ea152a2bd553 service nova] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:16.924311 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f621c5-1c86-4c15-9f69-ea152a2bd553 service nova] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:16.924311 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f621c5-1c86-4c15-9f69-ea152a2bd553 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:16.932805 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f621c5-1c86-4c15-9f69-ea152a2bd553 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:16.937863 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f621c5-1c86-4c15-9f69-ea152a2bd553 service nova] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:16.949250 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1166/2306] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:39:16 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:39:17.244572 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b0e0fb-b0fe-47cc-877e-f2eac6b0cb41 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:17.255059 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:39:17.351317 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b0e0fb-b0fe-47cc-877e-f2eac6b0cb41 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:17.356499 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7084f7f9-54fc-4ab1-a100-0262ddf57df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:17.357450 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084f7f9-54fc-4ab1-a100-0262ddf57df8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:17.357826 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084f7f9-54fc-4ab1-a100-0262ddf57df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:17.358177 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084f7f9-54fc-4ab1-a100-0262ddf57df8 admin admin] PATH_INFO: `/v3/users/23a33ea5c6014c7eaa9f766553245d50` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:17.361325 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7084f7f9-54fc-4ab1-a100-0262ddf57df8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23a33ea5c6014c7eaa9f766553245d50)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:17.361900 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7084f7f9-54fc-4ab1-a100-0262ddf57df8 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:17.523960 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-7084f7f9-54fc-4ab1-a100-0262ddf57df8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23a33ea5c6014c7eaa9f766553245d50'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:17.531844 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-7084f7f9-54fc-4ab1-a100-0262ddf57df8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23a33ea5c6014c7eaa9f766553245d50'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:17.548014 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-7084f7f9-54fc-4ab1-a100-0262ddf57df8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23a33ea5c6014c7eaa9f766553245d50'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:17.549008 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-7084f7f9-54fc-4ab1-a100-0262ddf57df8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23a33ea5c6014c7eaa9f766553245d50'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:17.621551 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-7084f7f9-54fc-4ab1-a100-0262ddf57df8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23a33ea5c6014c7eaa9f766553245d50'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:17.631634 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1141/2307] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:39:17 2022] DELETE /identity/v3/users/23a33ea5c6014c7eaa9f766553245d50 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:39:17.639736 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aded3f-aa28-46c8-badc-345eb83dbae8 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:17.639736 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aded3f-aa28-46c8-badc-345eb83dbae8 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:17.639736 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aded3f-aa28-46c8-badc-345eb83dbae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:17.656953 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-89aded3f-aa28-46c8-badc-345eb83dbae8 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:39:17.658253 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-89aded3f-aa28-46c8-badc-345eb83dbae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:17.724889 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1167/2308] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:39:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:39:17.736418 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7084f7f9-54fc-4ab1-a100-0262ddf57df8 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:17.737978 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-7084f7f9-54fc-4ab1-a100-0262ddf57df8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:17.807455 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7084f7f9-54fc-4ab1-a100-0262ddf57df8 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:17.811374 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b13d6-525f-417c-8e27-4dbc932eaf4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:17.812055 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b13d6-525f-417c-8e27-4dbc932eaf4d service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:17.812340 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b13d6-525f-417c-8e27-4dbc932eaf4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:17.812612 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b13d6-525f-417c-8e27-4dbc932eaf4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:17.823820 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590b13d6-525f-417c-8e27-4dbc932eaf4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:17.825264 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590b13d6-525f-417c-8e27-4dbc932eaf4d service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:17.836548 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1142/2309] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:39:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:39:18.264791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aded3f-aa28-46c8-badc-345eb83dbae8 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:18.266282 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-89aded3f-aa28-46c8-badc-345eb83dbae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:18.284385 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aded3f-aa28-46c8-badc-345eb83dbae8 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:18.293604 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da11ffb-a44d-4583-8126-26f0bba43fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:18.294610 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da11ffb-a44d-4583-8126-26f0bba43fe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:18.295712 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da11ffb-a44d-4583-8126-26f0bba43fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:18.295712 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da11ffb-a44d-4583-8126-26f0bba43fe7 admin admin] PATH_INFO: `/v3/projects/faef7e30953a41508d6f854702d13b0d` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:18.300268 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da11ffb-a44d-4583-8126-26f0bba43fe7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=faef7e30953a41508d6f854702d13b0d)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:18.302771 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da11ffb-a44d-4583-8126-26f0bba43fe7 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:18.349832 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:39:18.462908 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-1da11ffb-a44d-4583-8126-26f0bba43fe7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'faef7e30953a41508d6f854702d13b0d', 'request_id': 'req-1da11ffb-a44d-4583-8126-26f0bba43fe7'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:18.469178 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-1da11ffb-a44d-4583-8126-26f0bba43fe7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'faef7e30953a41508d6f854702d13b0d', 'request_id': 'req-1da11ffb-a44d-4583-8126-26f0bba43fe7'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:18.476756 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-1da11ffb-a44d-4583-8126-26f0bba43fe7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'faef7e30953a41508d6f854702d13b0d', 'request_id': 'req-1da11ffb-a44d-4583-8126-26f0bba43fe7'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:18.536291 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-1da11ffb-a44d-4583-8126-26f0bba43fe7 admin admin] The token cache is being invalidate because project faef7e30953a41508d6f854702d13b0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:39:18.536685 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-1da11ffb-a44d-4583-8126-26f0bba43fe7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:18.538655 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-1da11ffb-a44d-4583-8126-26f0bba43fe7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:18.550326 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1168/2310] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:39:18 2022] DELETE /identity/v3/projects/faef7e30953a41508d6f854702d13b0d => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:39:18.552126 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ded91-48af-4288-abae-f8e8487ddec4 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:18.552603 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ded91-48af-4288-abae-f8e8487ddec4 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:18.553077 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ded91-48af-4288-abae-f8e8487ddec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:18.573179 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-9d2ded91-48af-4288-abae-f8e8487ddec4 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:39:18.574645 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-9d2ded91-48af-4288-abae-f8e8487ddec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:18.602189 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:39:18.640906 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1143/2311] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:39:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:39:18.649591 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfe72d-a87e-48c7-93c2-9cb0833d811d None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:18.650523 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfe72d-a87e-48c7-93c2-9cb0833d811d None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:18.650812 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfe72d-a87e-48c7-93c2-9cb0833d811d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:18.671434 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-6ebfe72d-a87e-48c7-93c2-9cb0833d811d None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:39:18.672692 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-6ebfe72d-a87e-48c7-93c2-9cb0833d811d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:18.687362 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1169/2312] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:39:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:39:18.690307 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ded91-48af-4288-abae-f8e8487ddec4 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:18.705888 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ded91-48af-4288-abae-f8e8487ddec4 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:18.710325 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd640a13-642c-42a6-bc14-d188659c9f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:18.712100 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd640a13-642c-42a6-bc14-d188659c9f70 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:18.712385 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd640a13-642c-42a6-bc14-d188659c9f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:18.712665 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd640a13-642c-42a6-bc14-d188659c9f70 admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:18.713419 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd640a13-642c-42a6-bc14-d188659c9f70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:39:18.713894 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd640a13-642c-42a6-bc14-d188659c9f70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:18.721622 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd640a13-642c-42a6-bc14-d188659c9f70 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:18.732346 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1144/2313] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:39:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:39:18.735840 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebfe72d-a87e-48c7-93c2-9cb0833d811d None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:18.746301 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebfe72d-a87e-48c7-93c2-9cb0833d811d None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:18.751603 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53575d7-dda1-4d12-9279-6c535501d74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:18.752017 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53575d7-dda1-4d12-9279-6c535501d74b admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:18.752880 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53575d7-dda1-4d12-9279-6c535501d74b admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:18.753167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53575d7-dda1-4d12-9279-6c535501d74b admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:18.753872 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53575d7-dda1-4d12-9279-6c535501d74b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:18.755863 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53575d7-dda1-4d12-9279-6c535501d74b admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:18.864516 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1170/2314] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:39:18 2022] POST /identity/v3/projects => generated 383 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:39:18.869375 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd640a13-642c-42a6-bc14-d188659c9f70 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:18.880594 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd640a13-642c-42a6-bc14-d188659c9f70 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:18.945858 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a4518-91e4-4f87-8623-742ec1d9aca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:18.946337 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a4518-91e4-4f87-8623-742ec1d9aca0 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:18.946711 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a4518-91e4-4f87-8623-742ec1d9aca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:18.947047 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a4518-91e4-4f87-8623-742ec1d9aca0 admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:18.947854 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6a4518-91e4-4f87-8623-742ec1d9aca0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:18.949849 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6a4518-91e4-4f87-8623-742ec1d9aca0 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:19.063762 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1145/2315] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:39:18 2022] POST /identity/v3/users => generated 292 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:39:19.069024 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53575d7-dda1-4d12-9279-6c535501d74b admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:19.081242 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53575d7-dda1-4d12-9279-6c535501d74b admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:19.086405 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417bc5c2-53ee-4a3f-a096-ec3b23df4f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:19.087224 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417bc5c2-53ee-4a3f-a096-ec3b23df4f3e admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:19.087603 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417bc5c2-53ee-4a3f-a096-ec3b23df4f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:19.087603 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417bc5c2-53ee-4a3f-a096-ec3b23df4f3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:19.087603 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417bc5c2-53ee-4a3f-a096-ec3b23df4f3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:19.087603 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417bc5c2-53ee-4a3f-a096-ec3b23df4f3e admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:19.102690 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1171/2316] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:39:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:39:19.106848 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a4518-91e4-4f87-8623-742ec1d9aca0 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:19.119746 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a4518-91e4-4f87-8623-742ec1d9aca0 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:19.124767 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7475006-01fa-4cb6-9a39-debebf22abe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:19.129188 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7475006-01fa-4cb6-9a39-debebf22abe1 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:19.129188 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7475006-01fa-4cb6-9a39-debebf22abe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:19.129188 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7475006-01fa-4cb6-9a39-debebf22abe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:19.129188 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7475006-01fa-4cb6-9a39-debebf22abe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:19.129594 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7475006-01fa-4cb6-9a39-debebf22abe1 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:19.144797 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1146/2317] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:39:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:39:19.146108 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417bc5c2-53ee-4a3f-a096-ec3b23df4f3e admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:19.156548 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417bc5c2-53ee-4a3f-a096-ec3b23df4f3e admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:19.162440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61098465-8d09-4348-9d49-ce3990cb1e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:19.163480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61098465-8d09-4348-9d49-ce3990cb1e17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:19.163788 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61098465-8d09-4348-9d49-ce3990cb1e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:19.164061 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61098465-8d09-4348-9d49-ce3990cb1e17 admin admin] PATH_INFO: `/v3/projects/5ff53536524249eeb826bd997668151c/users/606ab54c6def4a6db681765d9f5a37c6/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:19.184093 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61098465-8d09-4348-9d49-ce3990cb1e17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ff53536524249eeb826bd997668151c, user_id=606ab54c6def4a6db681765d9f5a37c6, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:19.186751 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61098465-8d09-4348-9d49-ce3990cb1e17 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:19.526979 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1172/2318] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:39:19 2022] PUT /identity/v3/projects/5ff53536524249eeb826bd997668151c/users/606ab54c6def4a6db681765d9f5a37c6/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:39:19.540454 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2908fe0-4075-4fdc-851c-802d76e80626 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:19.540821 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2908fe0-4075-4fdc-851c-802d76e80626 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:19.541134 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2908fe0-4075-4fdc-851c-802d76e80626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:19.569392 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-f2908fe0-4075-4fdc-851c-802d76e80626 None None] MFA Rules not processed for user `606ab54c6def4a6db681765d9f5a37c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:39:19.570654 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-f2908fe0-4075-4fdc-851c-802d76e80626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:19.662341 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61098465-8d09-4348-9d49-ce3990cb1e17 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:19.663404 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1147/2319] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:39:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:39:19.750593 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61098465-8d09-4348-9d49-ce3990cb1e17 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:19.755801 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae311438-1034-4ffb-a434-6e3e5003d453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:19.756682 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae311438-1034-4ffb-a434-6e3e5003d453 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:19.756961 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae311438-1034-4ffb-a434-6e3e5003d453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:19.757235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae311438-1034-4ffb-a434-6e3e5003d453 admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:19.757928 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae311438-1034-4ffb-a434-6e3e5003d453 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:19.760028 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae311438-1034-4ffb-a434-6e3e5003d453 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:19.869308 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1173/2320] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:39:19 2022] POST /identity/v3/projects => generated 381 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:39:19.870776 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2908fe0-4075-4fdc-851c-802d76e80626 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:19.882650 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2908fe0-4075-4fdc-851c-802d76e80626 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:19.910100 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:39:19.960077 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e18e2e-0c54-441c-9de0-3e44f26cff62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:19.960679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e18e2e-0c54-441c-9de0-3e44f26cff62 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:19.960973 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e18e2e-0c54-441c-9de0-3e44f26cff62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:19.961267 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e18e2e-0c54-441c-9de0-3e44f26cff62 admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:19.962075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e18e2e-0c54-441c-9de0-3e44f26cff62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:19.965494 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e18e2e-0c54-441c-9de0-3e44f26cff62 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:20.044822 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1148/2321] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:39:19 2022] POST /identity/v3/users => generated 297 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:39:20.048499 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae311438-1034-4ffb-a434-6e3e5003d453 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:20.062917 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae311438-1034-4ffb-a434-6e3e5003d453 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:20.066873 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9dfc92-e7e0-40b7-b7f8-5e8ab270d0bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:20.067664 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9dfc92-e7e0-40b7-b7f8-5e8ab270d0bf admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:20.068017 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9dfc92-e7e0-40b7-b7f8-5e8ab270d0bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:20.068379 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9dfc92-e7e0-40b7-b7f8-5e8ab270d0bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:20.069186 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9dfc92-e7e0-40b7-b7f8-5e8ab270d0bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:20.070916 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9dfc92-e7e0-40b7-b7f8-5e8ab270d0bf admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:20.083508 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1174/2322] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:39:20.088104 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e18e2e-0c54-441c-9de0-3e44f26cff62 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:20.099410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e18e2e-0c54-441c-9de0-3e44f26cff62 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:20.104784 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759583c9-5a2a-49f2-8787-5b9c0dc1eee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:20.105772 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759583c9-5a2a-49f2-8787-5b9c0dc1eee5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:20.106189 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759583c9-5a2a-49f2-8787-5b9c0dc1eee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:20.106567 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759583c9-5a2a-49f2-8787-5b9c0dc1eee5 admin admin] PATH_INFO: `/v3/projects/d86761cd789342289667789ea43fcd0a/users/6f7c56168a37492ead7bb6a13f73a82a/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:20.123879 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759583c9-5a2a-49f2-8787-5b9c0dc1eee5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d86761cd789342289667789ea43fcd0a, user_id=6f7c56168a37492ead7bb6a13f73a82a, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:20.126691 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759583c9-5a2a-49f2-8787-5b9c0dc1eee5 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:20.204160 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1149/2323] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:39:20 2022] PUT /identity/v3/projects/d86761cd789342289667789ea43fcd0a/users/6f7c56168a37492ead7bb6a13f73a82a/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:39:20.208938 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9dfc92-e7e0-40b7-b7f8-5e8ab270d0bf admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:20.220220 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9dfc92-e7e0-40b7-b7f8-5e8ab270d0bf admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:20.240830 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:39:20.287914 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc646842-92e2-40f9-b358-8ab4cf687a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:20.288777 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc646842-92e2-40f9-b358-8ab4cf687a09 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:20.289145 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc646842-92e2-40f9-b358-8ab4cf687a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:20.289527 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc646842-92e2-40f9-b358-8ab4cf687a09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:20.290370 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc646842-92e2-40f9-b358-8ab4cf687a09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:20.292186 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc646842-92e2-40f9-b358-8ab4cf687a09 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:20.307869 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1175/2324] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:39:20.308596 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759583c9-5a2a-49f2-8787-5b9c0dc1eee5 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:20.321496 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759583c9-5a2a-49f2-8787-5b9c0dc1eee5 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:20.326658 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7e7f3-db1d-452b-976b-2b7251cf4b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:20.327543 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7e7f3-db1d-452b-976b-2b7251cf4b30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:20.327862 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7e7f3-db1d-452b-976b-2b7251cf4b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:20.328180 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7e7f3-db1d-452b-976b-2b7251cf4b30 admin admin] PATH_INFO: `/v3/projects/d86761cd789342289667789ea43fcd0a/users/6f7c56168a37492ead7bb6a13f73a82a/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:20.330158 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a7e7f3-db1d-452b-976b-2b7251cf4b30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d86761cd789342289667789ea43fcd0a, user_id=6f7c56168a37492ead7bb6a13f73a82a, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:20.335800 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a7e7f3-db1d-452b-976b-2b7251cf4b30 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:20.345805 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1150/2325] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:39:20 2022] PUT /identity/v3/projects/d86761cd789342289667789ea43fcd0a/users/6f7c56168a37492ead7bb6a13f73a82a/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:39:20.356440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad6f53f-5214-47ff-aaef-e863412c96f6 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:20.356844 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad6f53f-5214-47ff-aaef-e863412c96f6 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:20.357241 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad6f53f-5214-47ff-aaef-e863412c96f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:20.389299 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-3ad6f53f-5214-47ff-aaef-e863412c96f6 None None] MFA Rules not processed for user `6f7c56168a37492ead7bb6a13f73a82a`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:39:20.390538 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-3ad6f53f-5214-47ff-aaef-e863412c96f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:20.470770 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1176/2326] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:39:20 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:39:20.481939 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7e7f3-db1d-452b-976b-2b7251cf4b30 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:20.483788 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-e4a7e7f3-db1d-452b-976b-2b7251cf4b30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:20.595354 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7e7f3-db1d-452b-976b-2b7251cf4b30 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:20.600687 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02f3f2-7cd7-40c7-b01c-f4cf400c99d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:20.601450 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02f3f2-7cd7-40c7-b01c-f4cf400c99d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:20.601740 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02f3f2-7cd7-40c7-b01c-f4cf400c99d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:20.602022 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02f3f2-7cd7-40c7-b01c-f4cf400c99d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:20.614206 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa02f3f2-7cd7-40c7-b01c-f4cf400c99d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:20.615950 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa02f3f2-7cd7-40c7-b01c-f4cf400c99d6 service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:20.630841 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1151/2327] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:39:20 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:39:20.766985 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad6f53f-5214-47ff-aaef-e863412c96f6 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:20.779980 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad6f53f-5214-47ff-aaef-e863412c96f6 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:20.783843 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba64313-e894-4323-bc2b-445467ac6873 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:20.784989 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba64313-e894-4323-bc2b-445467ac6873 service cinder] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:20.785940 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba64313-e894-4323-bc2b-445467ac6873 service cinder] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:20.786317 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba64313-e894-4323-bc2b-445467ac6873 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:20.862035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba64313-e894-4323-bc2b-445467ac6873 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:20.863808 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba64313-e894-4323-bc2b-445467ac6873 service cinder] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:20.882615 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1177/2328] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:39:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:39:23.958166 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02f3f2-7cd7-40c7-b01c-f4cf400c99d6 service cinder] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:24.020571 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02f3f2-7cd7-40c7-b01c-f4cf400c99d6 service cinder] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:24.025006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d005e83-e1ec-4130-a415-4acd96520262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:24.025866 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d005e83-e1ec-4130-a415-4acd96520262 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:24.026147 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d005e83-e1ec-4130-a415-4acd96520262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:24.026417 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d005e83-e1ec-4130-a415-4acd96520262 admin admin] PATH_INFO: `/v3/users/606ab54c6def4a6db681765d9f5a37c6` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:24.027595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d005e83-e1ec-4130-a415-4acd96520262 admin admin] RBAC: Authorizing `identity:delete_user(user_id=606ab54c6def4a6db681765d9f5a37c6)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:24.029894 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d005e83-e1ec-4130-a415-4acd96520262 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:24.053745 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:39:24.269969 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-3d005e83-e1ec-4130-a415-4acd96520262 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '606ab54c6def4a6db681765d9f5a37c6'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:24.277125 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-3d005e83-e1ec-4130-a415-4acd96520262 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '606ab54c6def4a6db681765d9f5a37c6'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:24.292758 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-3d005e83-e1ec-4130-a415-4acd96520262 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '606ab54c6def4a6db681765d9f5a37c6'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:24.293817 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-3d005e83-e1ec-4130-a415-4acd96520262 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '606ab54c6def4a6db681765d9f5a37c6'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:24.364610 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-3d005e83-e1ec-4130-a415-4acd96520262 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '606ab54c6def4a6db681765d9f5a37c6'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:24.374172 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1152/2329] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:39:23 2022] DELETE /identity/v3/users/606ab54c6def4a6db681765d9f5a37c6 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:39:24.378041 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174dd72-6652-475e-acdf-e0578e37c1cd None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:24.378571 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174dd72-6652-475e-acdf-e0578e37c1cd None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:24.378846 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174dd72-6652-475e-acdf-e0578e37c1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:24.397893 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-0174dd72-6652-475e-acdf-e0578e37c1cd None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:39:24.400516 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-0174dd72-6652-475e-acdf-e0578e37c1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:24.447493 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:39:24.474266 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1178/2330] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:39:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:39:24.496994 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d005e83-e1ec-4130-a415-4acd96520262 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:24.498594 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-3d005e83-e1ec-4130-a415-4acd96520262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:24.567420 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d005e83-e1ec-4130-a415-4acd96520262 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:24.571599 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d4b955-3266-4209-a52c-7f2c5c307623 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:24.571861 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4b955-3266-4209-a52c-7f2c5c307623 service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:24.572149 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4b955-3266-4209-a52c-7f2c5c307623 service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:24.572425 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4b955-3266-4209-a52c-7f2c5c307623 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:24.581517 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d4b955-3266-4209-a52c-7f2c5c307623 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:24.583471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d4b955-3266-4209-a52c-7f2c5c307623 service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:24.592939 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1153/2331] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:39:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:39:25.024194 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0174dd72-6652-475e-acdf-e0578e37c1cd None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:25.025704 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-0174dd72-6652-475e-acdf-e0578e37c1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:25.037797 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0174dd72-6652-475e-acdf-e0578e37c1cd None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:25.043494 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9354cfe-35ef-43a5-afdd-d6519dedd6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:25.044694 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9354cfe-35ef-43a5-afdd-d6519dedd6d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:25.044981 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9354cfe-35ef-43a5-afdd-d6519dedd6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:25.045253 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9354cfe-35ef-43a5-afdd-d6519dedd6d2 admin admin] PATH_INFO: `/v3/projects/5ff53536524249eeb826bd997668151c` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:25.046447 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9354cfe-35ef-43a5-afdd-d6519dedd6d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ff53536524249eeb826bd997668151c)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:25.049467 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9354cfe-35ef-43a5-afdd-d6519dedd6d2 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:25.152308 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-d9354cfe-35ef-43a5-afdd-d6519dedd6d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ff53536524249eeb826bd997668151c', 'request_id': 'req-d9354cfe-35ef-43a5-afdd-d6519dedd6d2'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:25.157756 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-d9354cfe-35ef-43a5-afdd-d6519dedd6d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ff53536524249eeb826bd997668151c', 'request_id': 'req-d9354cfe-35ef-43a5-afdd-d6519dedd6d2'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:25.163753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-d9354cfe-35ef-43a5-afdd-d6519dedd6d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ff53536524249eeb826bd997668151c', 'request_id': 'req-d9354cfe-35ef-43a5-afdd-d6519dedd6d2'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:25.244749 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-d9354cfe-35ef-43a5-afdd-d6519dedd6d2 admin admin] The token cache is being invalidate because project 5ff53536524249eeb826bd997668151c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:39:25.245855 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-d9354cfe-35ef-43a5-afdd-d6519dedd6d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:25.245855 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-d9354cfe-35ef-43a5-afdd-d6519dedd6d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:25.247240 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1179/2332] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:39:25 2022] DELETE /identity/v3/projects/5ff53536524249eeb826bd997668151c => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:39:25.251288 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d4b955-3266-4209-a52c-7f2c5c307623 service neutron] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:25.262109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d4b955-3266-4209-a52c-7f2c5c307623 service neutron] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:25.318406 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d455a471-bd3f-4d1c-b237-8fd2d907877c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:25.319275 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455a471-bd3f-4d1c-b237-8fd2d907877c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:25.319555 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455a471-bd3f-4d1c-b237-8fd2d907877c admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:25.319860 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455a471-bd3f-4d1c-b237-8fd2d907877c admin admin] PATH_INFO: `/v3/users/6f7c56168a37492ead7bb6a13f73a82a` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:25.321034 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d455a471-bd3f-4d1c-b237-8fd2d907877c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f7c56168a37492ead7bb6a13f73a82a)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:25.323013 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d455a471-bd3f-4d1c-b237-8fd2d907877c admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:25.542202 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:39:25.544536 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-d455a471-bd3f-4d1c-b237-8fd2d907877c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f7c56168a37492ead7bb6a13f73a82a'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:25.553319 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-d455a471-bd3f-4d1c-b237-8fd2d907877c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f7c56168a37492ead7bb6a13f73a82a'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:25.565836 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-d455a471-bd3f-4d1c-b237-8fd2d907877c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f7c56168a37492ead7bb6a13f73a82a'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:25.567378 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-d455a471-bd3f-4d1c-b237-8fd2d907877c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f7c56168a37492ead7bb6a13f73a82a'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:25.638478 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-d455a471-bd3f-4d1c-b237-8fd2d907877c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f7c56168a37492ead7bb6a13f73a82a'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:25.645324 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1154/2333] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:39:25 2022] DELETE /identity/v3/users/6f7c56168a37492ead7bb6a13f73a82a => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:39:26.193165 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9354cfe-35ef-43a5-afdd-d6519dedd6d2 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:26.194849 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-d9354cfe-35ef-43a5-afdd-d6519dedd6d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:26.214173 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9354cfe-35ef-43a5-afdd-d6519dedd6d2 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:26.290482 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f77654-4957-429f-8459-32d4c7f89b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:26.291823 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f77654-4957-429f-8459-32d4c7f89b95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:26.293501 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f77654-4957-429f-8459-32d4c7f89b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:26.294012 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f77654-4957-429f-8459-32d4c7f89b95 admin admin] PATH_INFO: `/v3/projects/d86761cd789342289667789ea43fcd0a` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:26.295699 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f77654-4957-429f-8459-32d4c7f89b95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d86761cd789342289667789ea43fcd0a)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:26.298231 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f77654-4957-429f-8459-32d4c7f89b95 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:26.417813 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-62f77654-4957-429f-8459-32d4c7f89b95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd86761cd789342289667789ea43fcd0a', 'request_id': 'req-62f77654-4957-429f-8459-32d4c7f89b95'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:26.423489 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-62f77654-4957-429f-8459-32d4c7f89b95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd86761cd789342289667789ea43fcd0a', 'request_id': 'req-62f77654-4957-429f-8459-32d4c7f89b95'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:26.430102 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-62f77654-4957-429f-8459-32d4c7f89b95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd86761cd789342289667789ea43fcd0a', 'request_id': 'req-62f77654-4957-429f-8459-32d4c7f89b95'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:26.433926 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:39:26.517432 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-62f77654-4957-429f-8459-32d4c7f89b95 admin admin] The token cache is being invalidate because project d86761cd789342289667789ea43fcd0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:39:26.517699 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-62f77654-4957-429f-8459-32d4c7f89b95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:26.518375 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-62f77654-4957-429f-8459-32d4c7f89b95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:26.520809 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1180/2334] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:39:26 2022] DELETE /identity/v3/projects/d86761cd789342289667789ea43fcd0a => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:39:26.525726 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34eabdf-fa9a-40b8-ab1b-0d2a31f245f4 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:26.526178 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34eabdf-fa9a-40b8-ab1b-0d2a31f245f4 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:26.526288 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34eabdf-fa9a-40b8-ab1b-0d2a31f245f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:26.544965 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-e34eabdf-fa9a-40b8-ab1b-0d2a31f245f4 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:39:26.546214 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-e34eabdf-fa9a-40b8-ab1b-0d2a31f245f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:26.609249 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1155/2335] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:39:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:39:26.619189 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9438a8-fb5b-4c3e-a4b6-3ea955596653 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:26.619353 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9438a8-fb5b-4c3e-a4b6-3ea955596653 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:26.619663 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9438a8-fb5b-4c3e-a4b6-3ea955596653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:26.638743 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-bd9438a8-fb5b-4c3e-a4b6-3ea955596653 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:39:26.640086 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-bd9438a8-fb5b-4c3e-a4b6-3ea955596653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:26.648277 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1181/2336] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:39:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:39:26.652435 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34eabdf-fa9a-40b8-ab1b-0d2a31f245f4 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:26.662197 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34eabdf-fa9a-40b8-ab1b-0d2a31f245f4 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:26.667662 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347134e-ad33-4b0e-9804-02570d724064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:26.668337 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347134e-ad33-4b0e-9804-02570d724064 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:26.668646 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347134e-ad33-4b0e-9804-02570d724064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:26.668924 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347134e-ad33-4b0e-9804-02570d724064 admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:26.669692 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347134e-ad33-4b0e-9804-02570d724064 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:39:26.670004 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347134e-ad33-4b0e-9804-02570d724064 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:26.671617 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347134e-ad33-4b0e-9804-02570d724064 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:26.684847 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1156/2337] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:39:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:39:26.689186 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9438a8-fb5b-4c3e-a4b6-3ea955596653 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:26.699063 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9438a8-fb5b-4c3e-a4b6-3ea955596653 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:26.703723 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c056736-0176-45a8-b261-60733700254a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:26.704389 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c056736-0176-45a8-b261-60733700254a admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:26.704669 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c056736-0176-45a8-b261-60733700254a admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:26.704942 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c056736-0176-45a8-b261-60733700254a admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:26.705636 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c056736-0176-45a8-b261-60733700254a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:26.707601 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c056736-0176-45a8-b261-60733700254a admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:26.800428 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1182/2338] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:39:26 2022] POST /identity/v3/projects => generated 365 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:39:26.804986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347134e-ad33-4b0e-9804-02570d724064 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:26.815221 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347134e-ad33-4b0e-9804-02570d724064 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:26.872276 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae38b7c3-cb8f-4ced-9a69-835be1d73235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:26.872973 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38b7c3-cb8f-4ced-9a69-835be1d73235 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:26.873253 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38b7c3-cb8f-4ced-9a69-835be1d73235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:26.873531 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38b7c3-cb8f-4ced-9a69-835be1d73235 admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:26.874226 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae38b7c3-cb8f-4ced-9a69-835be1d73235 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:26.876149 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae38b7c3-cb8f-4ced-9a69-835be1d73235 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:26.958726 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1157/2339] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:39:26 2022] POST /identity/v3/users => generated 283 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:39:26.962869 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c056736-0176-45a8-b261-60733700254a admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:26.973961 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c056736-0176-45a8-b261-60733700254a admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:26.978554 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f9bfa-3931-422c-b4b2-4773e8e0697d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:26.979193 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f9bfa-3931-422c-b4b2-4773e8e0697d admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:26.979468 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f9bfa-3931-422c-b4b2-4773e8e0697d admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:26.979770 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f9bfa-3931-422c-b4b2-4773e8e0697d admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:26.980490 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101f9bfa-3931-422c-b4b2-4773e8e0697d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:26.982062 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101f9bfa-3931-422c-b4b2-4773e8e0697d admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:26.992679 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1183/2340] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:39:26.997593 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae38b7c3-cb8f-4ced-9a69-835be1d73235 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:27.007708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae38b7c3-cb8f-4ced-9a69-835be1d73235 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:27.012268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfd2486-b84f-43f2-ab0d-c532854cf126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:27.012910 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd2486-b84f-43f2-ab0d-c532854cf126 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:27.013190 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd2486-b84f-43f2-ab0d-c532854cf126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:27.013460 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd2486-b84f-43f2-ab0d-c532854cf126 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:27.014174 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfd2486-b84f-43f2-ab0d-c532854cf126 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:27.015781 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfd2486-b84f-43f2-ab0d-c532854cf126 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:27.025472 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1158/2341] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:39:27.029429 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f9bfa-3931-422c-b4b2-4773e8e0697d admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:27.039825 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f9bfa-3931-422c-b4b2-4773e8e0697d admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:27.044418 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b80684-5eb3-43de-b390-b9d15dee5025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:27.045269 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b80684-5eb3-43de-b390-b9d15dee5025 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:27.045548 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b80684-5eb3-43de-b390-b9d15dee5025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:27.045820 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b80684-5eb3-43de-b390-b9d15dee5025 admin admin] PATH_INFO: `/v3/projects/8eeeb7f7e3c04985beb9659d126044dc/users/60209b4aa7d1455aaeb831c6884cefc5/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:27.060704 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b80684-5eb3-43de-b390-b9d15dee5025 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eeeb7f7e3c04985beb9659d126044dc, user_id=60209b4aa7d1455aaeb831c6884cefc5, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:27.063257 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b80684-5eb3-43de-b390-b9d15dee5025 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:27.124375 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1184/2342] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:39:27 2022] PUT /identity/v3/projects/8eeeb7f7e3c04985beb9659d126044dc/users/60209b4aa7d1455aaeb831c6884cefc5/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:39:27.134629 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc0711-5063-4a0b-833a-a35281d78c8f None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:27.134915 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc0711-5063-4a0b-833a-a35281d78c8f None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:27.135192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc0711-5063-4a0b-833a-a35281d78c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:27.161868 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-06cc0711-5063-4a0b-833a-a35281d78c8f None None] MFA Rules not processed for user `60209b4aa7d1455aaeb831c6884cefc5`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:39:27.163151 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-06cc0711-5063-4a0b-833a-a35281d78c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:27.176439 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:39:27.227342 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1159/2343] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:39:27 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:39:27.231890 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b80684-5eb3-43de-b390-b9d15dee5025 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:27.293200 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b80684-5eb3-43de-b390-b9d15dee5025 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:27.297770 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5dd1f0-535a-454b-99f9-874072f45b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:27.298419 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5dd1f0-535a-454b-99f9-874072f45b75 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:27.298698 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5dd1f0-535a-454b-99f9-874072f45b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:27.298976 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5dd1f0-535a-454b-99f9-874072f45b75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:27.299737 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5dd1f0-535a-454b-99f9-874072f45b75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:27.302255 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5dd1f0-535a-454b-99f9-874072f45b75 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:27.372502 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1185/2344] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:39:27 2022] POST /identity/v3/projects => generated 367 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:39:27.389320 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc0711-5063-4a0b-833a-a35281d78c8f None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:27.400534 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc0711-5063-4a0b-833a-a35281d78c8f None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:27.458131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf7dd5-d70e-4bf7-ae92-37b568c8dd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:27.458796 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf7dd5-d70e-4bf7-ae92-37b568c8dd3e admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:27.459079 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf7dd5-d70e-4bf7-ae92-37b568c8dd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:27.459357 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf7dd5-d70e-4bf7-ae92-37b568c8dd3e admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:27.460119 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bf7dd5-d70e-4bf7-ae92-37b568c8dd3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:27.462126 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bf7dd5-d70e-4bf7-ae92-37b568c8dd3e admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:27.542495 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1160/2345] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:39:27 2022] POST /identity/v3/users => generated 290 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:39:27.546426 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5dd1f0-535a-454b-99f9-874072f45b75 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:27.556748 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5dd1f0-535a-454b-99f9-874072f45b75 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:27.561477 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342900f3-0681-4eea-8cb1-3facdf4fa6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:27.562128 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342900f3-0681-4eea-8cb1-3facdf4fa6da admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:27.562409 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342900f3-0681-4eea-8cb1-3facdf4fa6da admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:27.562736 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342900f3-0681-4eea-8cb1-3facdf4fa6da admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:27.563459 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342900f3-0681-4eea-8cb1-3facdf4fa6da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:27.565134 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342900f3-0681-4eea-8cb1-3facdf4fa6da admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:27.575675 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1186/2346] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:39:27.579988 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf7dd5-d70e-4bf7-ae92-37b568c8dd3e admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:27.590438 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf7dd5-d70e-4bf7-ae92-37b568c8dd3e admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:27.595028 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d1e95-e653-4d8d-b99e-c941617689c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:27.595923 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d1e95-e653-4d8d-b99e-c941617689c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:27.596201 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d1e95-e653-4d8d-b99e-c941617689c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:27.596473 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d1e95-e653-4d8d-b99e-c941617689c0 admin admin] PATH_INFO: `/v3/projects/38c1f2ff455e4680b9c0e4fd5cbd0ba9/users/7608d8890d39463796c95a5e75c97573/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:27.612186 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d1e95-e653-4d8d-b99e-c941617689c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38c1f2ff455e4680b9c0e4fd5cbd0ba9, user_id=7608d8890d39463796c95a5e75c97573, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:27.615075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d1e95-e653-4d8d-b99e-c941617689c0 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:27.708579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1161/2347] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:39:27 2022] PUT /identity/v3/projects/38c1f2ff455e4680b9c0e4fd5cbd0ba9/users/7608d8890d39463796c95a5e75c97573/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:39:27.712477 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342900f3-0681-4eea-8cb1-3facdf4fa6da admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:27.722470 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342900f3-0681-4eea-8cb1-3facdf4fa6da admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:27.780051 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:39:27.784615 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a062310e-a6ea-4ea0-bb10-acec90ca8230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:27.785493 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062310e-a6ea-4ea0-bb10-acec90ca8230 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:27.785716 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062310e-a6ea-4ea0-bb10-acec90ca8230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:27.786001 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062310e-a6ea-4ea0-bb10-acec90ca8230 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:27.786737 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a062310e-a6ea-4ea0-bb10-acec90ca8230 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:27.788381 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a062310e-a6ea-4ea0-bb10-acec90ca8230 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:27.808229 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1187/2348] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:39:27.808581 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d1e95-e653-4d8d-b99e-c941617689c0 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:27.821675 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d1e95-e653-4d8d-b99e-c941617689c0 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:27.828106 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a36cc-6d9c-4f16-89eb-5cacacb7adce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:27.829091 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a36cc-6d9c-4f16-89eb-5cacacb7adce admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:27.829430 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a36cc-6d9c-4f16-89eb-5cacacb7adce admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:27.829796 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a36cc-6d9c-4f16-89eb-5cacacb7adce admin admin] PATH_INFO: `/v3/projects/38c1f2ff455e4680b9c0e4fd5cbd0ba9/users/7608d8890d39463796c95a5e75c97573/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:27.832434 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65a36cc-6d9c-4f16-89eb-5cacacb7adce admin admin] RBAC: Authorizing `identity:create_grant(project_id=38c1f2ff455e4680b9c0e4fd5cbd0ba9, user_id=7608d8890d39463796c95a5e75c97573, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:27.835508 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65a36cc-6d9c-4f16-89eb-5cacacb7adce admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:27.847902 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1162/2349] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:39:27 2022] PUT /identity/v3/projects/38c1f2ff455e4680b9c0e4fd5cbd0ba9/users/7608d8890d39463796c95a5e75c97573/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:39:27.857797 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feaa2f4-e6a2-42b1-beed-da773e2071cc None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:27.858087 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feaa2f4-e6a2-42b1-beed-da773e2071cc None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:27.858366 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feaa2f4-e6a2-42b1-beed-da773e2071cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:27.888726 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-7feaa2f4-e6a2-42b1-beed-da773e2071cc None None] MFA Rules not processed for user `7608d8890d39463796c95a5e75c97573`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:39:27.889817 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-7feaa2f4-e6a2-42b1-beed-da773e2071cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:27.969949 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1188/2350] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:39:27 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:39:27.981880 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a36cc-6d9c-4f16-89eb-5cacacb7adce admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:27.983843 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-b65a36cc-6d9c-4f16-89eb-5cacacb7adce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:28.056387 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a36cc-6d9c-4f16-89eb-5cacacb7adce admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:28.060257 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf38da-60c5-4793-8c6e-7e6a9d64dd8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:28.060932 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf38da-60c5-4793-8c6e-7e6a9d64dd8e service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:28.061219 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf38da-60c5-4793-8c6e-7e6a9d64dd8e service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:28.061504 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf38da-60c5-4793-8c6e-7e6a9d64dd8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:28.071538 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daf38da-60c5-4793-8c6e-7e6a9d64dd8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:28.072943 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daf38da-60c5-4793-8c6e-7e6a9d64dd8e service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:28.082426 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1163/2351] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:39:27 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:39:28.292241 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feaa2f4-e6a2-42b1-beed-da773e2071cc None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:28.302148 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feaa2f4-e6a2-42b1-beed-da773e2071cc None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:28.305689 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf854eb-37aa-4d0d-a8d6-e4e52922b88d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:28.306321 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf854eb-37aa-4d0d-a8d6-e4e52922b88d service cinder] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:28.306601 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf854eb-37aa-4d0d-a8d6-e4e52922b88d service cinder] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:28.306875 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf854eb-37aa-4d0d-a8d6-e4e52922b88d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:28.364121 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf854eb-37aa-4d0d-a8d6-e4e52922b88d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:28.365555 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf854eb-37aa-4d0d-a8d6-e4e52922b88d service cinder] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:28.375934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1189/2352] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:39:28 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:39:31.989298 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84ba3b-f7a5-4aa9-9bd5-e75bef86853b None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:31.989874 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84ba3b-f7a5-4aa9-9bd5-e75bef86853b None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:31.989960 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84ba3b-f7a5-4aa9-9bd5-e75bef86853b None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:31.991266 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1164/2353] 10.222.0.25 () {54 vars in 921 bytes} [Thu Jul 7 15:39:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:39:31.996934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cddaee-de22-477d-98bd-6b6c0eb1aac2 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:31.997417 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cddaee-de22-477d-98bd-6b6c0eb1aac2 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:31.997606 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cddaee-de22-477d-98bd-6b6c0eb1aac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:32.018393 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-03cddaee-de22-477d-98bd-6b6c0eb1aac2 None None] MFA Rules not processed for user `de1632555cb345f2b9213da1925c02f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:39:32.018393 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-03cddaee-de22-477d-98bd-6b6c0eb1aac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:32.094352 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1190/2354] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Jul 7 15:39:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:39:32.106346 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb84ba3b-f7a5-4aa9-9bd5-e75bef86853b None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:32.108067 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-bb84ba3b-f7a5-4aa9-9bd5-e75bef86853b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:32.121895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:39:32.189980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb84ba3b-f7a5-4aa9-9bd5-e75bef86853b None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:32.194487 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94ed85a-c756-4704-8590-969491a21605 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0d29e718f446e2a9fd1b2575a86041', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:32.195189 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ed85a-c756-4704-8590-969491a21605 service glance] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:32.195521 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ed85a-c756-4704-8590-969491a21605 service glance] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:32.201882 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ed85a-c756-4704-8590-969491a21605 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:32.205903 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94ed85a-c756-4704-8590-969491a21605 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:32.207310 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94ed85a-c756-4704-8590-969491a21605 service glance] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:32.218430 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1165/2355] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:39:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:39:32.406168 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687d30ce-82bb-4cbe-b87c-261c9e41574e None None] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:32.406455 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687d30ce-82bb-4cbe-b87c-261c9e41574e None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:32.406752 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687d30ce-82bb-4cbe-b87c-261c9e41574e None None] PATH_INFO: `/` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:32.408152 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1191/2356] 10.222.0.25 () {54 vars in 921 bytes} [Thu Jul 7 15:39:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 15:39:32.413470 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656561c1-d960-47c6-88b8-b5e165be101d None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:32.413765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656561c1-d960-47c6-88b8-b5e165be101d None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:32.414048 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656561c1-d960-47c6-88b8-b5e165be101d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:32.449116 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-656561c1-d960-47c6-88b8-b5e165be101d None None] MFA Rules not processed for user `e79efaea155c46ecb9831956e68aec8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:39:32.450348 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-656561c1-d960-47c6-88b8-b5e165be101d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:32.517946 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1166/2357] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Jul 7 15:39:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:39:32.530609 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687d30ce-82bb-4cbe-b87c-261c9e41574e None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:32.532293 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-687d30ce-82bb-4cbe-b87c-261c9e41574e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:32.546104 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687d30ce-82bb-4cbe-b87c-261c9e41574e None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:32.550328 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61589450-59b6-402a-8079-3345303226fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:32.550900 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61589450-59b6-402a-8079-3345303226fb service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:32.551181 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61589450-59b6-402a-8079-3345303226fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:32.551457 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61589450-59b6-402a-8079-3345303226fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:32.557326 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:39:32.560001 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61589450-59b6-402a-8079-3345303226fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:32.561416 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61589450-59b6-402a-8079-3345303226fb service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:32.569994 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1192/2358] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:39:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:39:32.661065 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e3fb5-ce99-4af7-a53e-0ed56344c8b1 None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:32.661389 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e3fb5-ce99-4af7-a53e-0ed56344c8b1 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:32.661729 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e3fb5-ce99-4af7-a53e-0ed56344c8b1 None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:32.663163 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1167/2359] 10.222.0.25 () {54 vars in 921 bytes} [Thu Jul 7 15:39:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:39:32.668463 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd397e-3cc8-4cfc-b5a2-0c188eab7741 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:32.668753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd397e-3cc8-4cfc-b5a2-0c188eab7741 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:32.669032 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd397e-3cc8-4cfc-b5a2-0c188eab7741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:32.688338 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-07cd397e-3cc8-4cfc-b5a2-0c188eab7741 None None] MFA Rules not processed for user `de1632555cb345f2b9213da1925c02f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:39:32.689541 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-07cd397e-3cc8-4cfc-b5a2-0c188eab7741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:32.700217 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1193/2360] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Jul 7 15:39:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:39:32.711125 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0e3fb5-ce99-4af7-a53e-0ed56344c8b1 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:32.721880 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0e3fb5-ce99-4af7-a53e-0ed56344c8b1 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:32.725458 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa6ea6-8b38-40be-a21c-6db4fce87208 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:32.726118 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa6ea6-8b38-40be-a21c-6db4fce87208 service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:32.726401 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa6ea6-8b38-40be-a21c-6db4fce87208 service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:32.726676 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa6ea6-8b38-40be-a21c-6db4fce87208 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:32.737502 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aa6ea6-8b38-40be-a21c-6db4fce87208 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:32.738928 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aa6ea6-8b38-40be-a21c-6db4fce87208 service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:32.750966 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1168/2361] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:39:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:39:34.949591 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cd397e-3cc8-4cfc-b5a2-0c188eab7741 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:35.012911 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cd397e-3cc8-4cfc-b5a2-0c188eab7741 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:35.020318 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561ad73-122d-42ee-ad15-b6ea962db165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:35.020318 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561ad73-122d-42ee-ad15-b6ea962db165 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:35.020318 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561ad73-122d-42ee-ad15-b6ea962db165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:35.020318 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561ad73-122d-42ee-ad15-b6ea962db165 admin admin] PATH_INFO: `/v3/users/60209b4aa7d1455aaeb831c6884cefc5` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:35.020738 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6561ad73-122d-42ee-ad15-b6ea962db165 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60209b4aa7d1455aaeb831c6884cefc5)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:35.022092 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6561ad73-122d-42ee-ad15-b6ea962db165 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:35.096936 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb109a0b-c097-4909-8e9c-c14d851db1a2 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:35.097389 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb109a0b-c097-4909-8e9c-c14d851db1a2 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:35.097503 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb109a0b-c097-4909-8e9c-c14d851db1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:35.117805 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-cb109a0b-c097-4909-8e9c-c14d851db1a2 None None] MFA Rules not processed for user `cd3c04f39d774998b66e6ce920925691`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:39:35.117805 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-cb109a0b-c097-4909-8e9c-c14d851db1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:35.173385 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1169/2362] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Jul 7 15:39:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:39:35.186452 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb109a0b-c097-4909-8e9c-c14d851db1a2 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:35.188645 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-cb109a0b-c097-4909-8e9c-c14d851db1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:35.192538 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-6561ad73-122d-42ee-ad15-b6ea962db165 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60209b4aa7d1455aaeb831c6884cefc5'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:35.193161 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-6561ad73-122d-42ee-ad15-b6ea962db165 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60209b4aa7d1455aaeb831c6884cefc5'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:35.200947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-6561ad73-122d-42ee-ad15-b6ea962db165 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60209b4aa7d1455aaeb831c6884cefc5'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:35.202132 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-6561ad73-122d-42ee-ad15-b6ea962db165 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60209b4aa7d1455aaeb831c6884cefc5'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:35.215227 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-6561ad73-122d-42ee-ad15-b6ea962db165 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60209b4aa7d1455aaeb831c6884cefc5'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:35.233248 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:39:35.263598 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb109a0b-c097-4909-8e9c-c14d851db1a2 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:35.268864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef564062-3dd7-433f-96d0-13ad26ebbfd7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b18033f849b240999d12c1885d2f7b91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:35.269531 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef564062-3dd7-433f-96d0-13ad26ebbfd7 service swift] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:35.269812 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef564062-3dd7-433f-96d0-13ad26ebbfd7 service swift] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:35.270087 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef564062-3dd7-433f-96d0-13ad26ebbfd7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:35.272463 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-ef564062-3dd7-433f-96d0-13ad26ebbfd7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:35.284072 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef564062-3dd7-433f-96d0-13ad26ebbfd7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:35.285727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef564062-3dd7-433f-96d0-13ad26ebbfd7 service swift] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:35.295718 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1170/2363] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Jul 7 15:39:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 07 15:39:35.301865 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1194/2364] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:39:34 2022] DELETE /identity/v3/users/60209b4aa7d1455aaeb831c6884cefc5 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:39:35.317253 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489cbf34-ec2b-45be-97db-0e0b84564367 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:35.317253 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489cbf34-ec2b-45be-97db-0e0b84564367 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:35.317253 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489cbf34-ec2b-45be-97db-0e0b84564367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:35.325703 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-489cbf34-ec2b-45be-97db-0e0b84564367 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:39:35.326904 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-489cbf34-ec2b-45be-97db-0e0b84564367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:35.385460 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1171/2365] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:39:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:39:35.395038 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561ad73-122d-42ee-ad15-b6ea962db165 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:35.396485 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-6561ad73-122d-42ee-ad15-b6ea962db165 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:35.467179 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561ad73-122d-42ee-ad15-b6ea962db165 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:35.474439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351d5da-ea90-4daf-8931-33ed2f035b6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:35.475445 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351d5da-ea90-4daf-8931-33ed2f035b6e service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:35.475986 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351d5da-ea90-4daf-8931-33ed2f035b6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:35.476853 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351d5da-ea90-4daf-8931-33ed2f035b6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:35.493940 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0351d5da-ea90-4daf-8931-33ed2f035b6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:35.497283 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0351d5da-ea90-4daf-8931-33ed2f035b6e service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:35.516433 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1195/2366] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:39:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:39:36.044751 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489cbf34-ec2b-45be-97db-0e0b84564367 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:36.046237 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-489cbf34-ec2b-45be-97db-0e0b84564367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:36.065815 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489cbf34-ec2b-45be-97db-0e0b84564367 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:36.083924 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23526f-9a7d-4e9b-b54a-c931ad38b2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:36.084823 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23526f-9a7d-4e9b-b54a-c931ad38b2be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:36.085107 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23526f-9a7d-4e9b-b54a-c931ad38b2be admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:36.085382 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23526f-9a7d-4e9b-b54a-c931ad38b2be admin admin] PATH_INFO: `/v3/projects/8eeeb7f7e3c04985beb9659d126044dc` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:36.086672 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c23526f-9a7d-4e9b-b54a-c931ad38b2be admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eeeb7f7e3c04985beb9659d126044dc)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:36.089206 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c23526f-9a7d-4e9b-b54a-c931ad38b2be admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:36.224066 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-5c23526f-9a7d-4e9b-b54a-c931ad38b2be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eeeb7f7e3c04985beb9659d126044dc', 'request_id': 'req-5c23526f-9a7d-4e9b-b54a-c931ad38b2be'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:36.232891 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-5c23526f-9a7d-4e9b-b54a-c931ad38b2be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eeeb7f7e3c04985beb9659d126044dc', 'request_id': 'req-5c23526f-9a7d-4e9b-b54a-c931ad38b2be'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:36.241727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-5c23526f-9a7d-4e9b-b54a-c931ad38b2be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eeeb7f7e3c04985beb9659d126044dc', 'request_id': 'req-5c23526f-9a7d-4e9b-b54a-c931ad38b2be'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:36.326429 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-5c23526f-9a7d-4e9b-b54a-c931ad38b2be admin admin] The token cache is being invalidate because project 8eeeb7f7e3c04985beb9659d126044dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:39:36.326852 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-5c23526f-9a7d-4e9b-b54a-c931ad38b2be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:36.327500 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-5c23526f-9a7d-4e9b-b54a-c931ad38b2be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:36.329439 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1172/2367] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:39:36 2022] DELETE /identity/v3/projects/8eeeb7f7e3c04985beb9659d126044dc => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:39:36.333079 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351d5da-ea90-4daf-8931-33ed2f035b6e service neutron] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:36.347880 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351d5da-ea90-4daf-8931-33ed2f035b6e service neutron] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:36.360475 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:39:36.437071 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d807db06-9b6f-4a54-a9c4-381ec5311acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:36.437777 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807db06-9b6f-4a54-a9c4-381ec5311acf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:36.438230 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807db06-9b6f-4a54-a9c4-381ec5311acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:36.438518 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807db06-9b6f-4a54-a9c4-381ec5311acf admin admin] PATH_INFO: `/v3/users/7608d8890d39463796c95a5e75c97573` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:36.439978 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d807db06-9b6f-4a54-a9c4-381ec5311acf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7608d8890d39463796c95a5e75c97573)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:36.441951 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d807db06-9b6f-4a54-a9c4-381ec5311acf admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:36.620714 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-d807db06-9b6f-4a54-a9c4-381ec5311acf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7608d8890d39463796c95a5e75c97573'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:36.623014 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-d807db06-9b6f-4a54-a9c4-381ec5311acf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7608d8890d39463796c95a5e75c97573'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:36.639508 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-d807db06-9b6f-4a54-a9c4-381ec5311acf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7608d8890d39463796c95a5e75c97573'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:36.640458 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-d807db06-9b6f-4a54-a9c4-381ec5311acf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7608d8890d39463796c95a5e75c97573'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:36.653807 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-d807db06-9b6f-4a54-a9c4-381ec5311acf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7608d8890d39463796c95a5e75c97573'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:36.725523 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1196/2368] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:39:36 2022] DELETE /identity/v3/users/7608d8890d39463796c95a5e75c97573 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:39:37.280157 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23526f-9a7d-4e9b-b54a-c931ad38b2be admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:37.281683 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-5c23526f-9a7d-4e9b-b54a-c931ad38b2be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:37.295306 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23526f-9a7d-4e9b-b54a-c931ad38b2be admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:37.355188 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:39:37.372167 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a2c049-eb0d-4fae-8dd6-b7b37ce03d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:37.372962 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2c049-eb0d-4fae-8dd6-b7b37ce03d49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:37.373247 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2c049-eb0d-4fae-8dd6-b7b37ce03d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:37.373518 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2c049-eb0d-4fae-8dd6-b7b37ce03d49 admin admin] PATH_INFO: `/v3/projects/38c1f2ff455e4680b9c0e4fd5cbd0ba9` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:37.374734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a2c049-eb0d-4fae-8dd6-b7b37ce03d49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38c1f2ff455e4680b9c0e4fd5cbd0ba9)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:37.377038 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a2c049-eb0d-4fae-8dd6-b7b37ce03d49 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:37.480851 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-13a2c049-eb0d-4fae-8dd6-b7b37ce03d49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38c1f2ff455e4680b9c0e4fd5cbd0ba9', 'request_id': 'req-13a2c049-eb0d-4fae-8dd6-b7b37ce03d49'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:37.487130 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-13a2c049-eb0d-4fae-8dd6-b7b37ce03d49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38c1f2ff455e4680b9c0e4fd5cbd0ba9', 'request_id': 'req-13a2c049-eb0d-4fae-8dd6-b7b37ce03d49'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:37.493745 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-13a2c049-eb0d-4fae-8dd6-b7b37ce03d49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38c1f2ff455e4680b9c0e4fd5cbd0ba9', 'request_id': 'req-13a2c049-eb0d-4fae-8dd6-b7b37ce03d49'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:37.563184 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-13a2c049-eb0d-4fae-8dd6-b7b37ce03d49 admin admin] The token cache is being invalidate because project 38c1f2ff455e4680b9c0e4fd5cbd0ba9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:39:37.563469 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-13a2c049-eb0d-4fae-8dd6-b7b37ce03d49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:37.564158 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-13a2c049-eb0d-4fae-8dd6-b7b37ce03d49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:39:37.565688 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1173/2369] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:39:37 2022] DELETE /identity/v3/projects/38c1f2ff455e4680b9c0e4fd5cbd0ba9 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:39:37.576203 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06784ded-ddb3-4c41-8826-edc3be5c5a23 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:37.576739 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06784ded-ddb3-4c41-8826-edc3be5c5a23 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:37.576860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06784ded-ddb3-4c41-8826-edc3be5c5a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:37.595102 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-06784ded-ddb3-4c41-8826-edc3be5c5a23 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:39:37.596350 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-06784ded-ddb3-4c41-8826-edc3be5c5a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:37.688797 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1197/2370] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:39:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:39:37.699189 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65face-5236-4283-aa8e-faed68b9690d None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:37.699807 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65face-5236-4283-aa8e-faed68b9690d None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:37.706485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65face-5236-4283-aa8e-faed68b9690d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:37.751589 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-1b65face-5236-4283-aa8e-faed68b9690d None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:39:37.751589 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-1b65face-5236-4283-aa8e-faed68b9690d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:37.765510 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06784ded-ddb3-4c41-8826-edc3be5c5a23 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:37.766048 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1174/2371] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:39:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:39:37.778710 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06784ded-ddb3-4c41-8826-edc3be5c5a23 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:37.786137 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ca042-2582-43ad-8608-01e7d3d4672f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:37.787087 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ca042-2582-43ad-8608-01e7d3d4672f admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:37.787472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ca042-2582-43ad-8608-01e7d3d4672f admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:37.787969 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ca042-2582-43ad-8608-01e7d3d4672f admin admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:37.788830 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ca042-2582-43ad-8608-01e7d3d4672f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:39:37.789213 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ca042-2582-43ad-8608-01e7d3d4672f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:37.790935 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ca042-2582-43ad-8608-01e7d3d4672f admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:37.804604 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1198/2372] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:39:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:39:37.808977 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65face-5236-4283-aa8e-faed68b9690d None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:37.818820 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65face-5236-4283-aa8e-faed68b9690d None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:37.824716 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee3ed28-5f0e-44b8-893d-7082c8e92af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:37.825366 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3ed28-5f0e-44b8-893d-7082c8e92af1 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:37.825649 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3ed28-5f0e-44b8-893d-7082c8e92af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:37.825924 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3ed28-5f0e-44b8-893d-7082c8e92af1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:37.827346 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee3ed28-5f0e-44b8-893d-7082c8e92af1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:37.831682 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee3ed28-5f0e-44b8-893d-7082c8e92af1 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:37.970130 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1175/2373] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:39:37 2022] POST /identity/v3/projects => generated 391 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:39:37.974378 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ca042-2582-43ad-8608-01e7d3d4672f admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:37.985609 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ca042-2582-43ad-8608-01e7d3d4672f admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:38.000110 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:39:38.050679 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73546db3-6982-4f81-8479-694e70a42439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:38.051451 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73546db3-6982-4f81-8479-694e70a42439 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:38.052386 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73546db3-6982-4f81-8479-694e70a42439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:38.052738 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73546db3-6982-4f81-8479-694e70a42439 admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:38.053513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73546db3-6982-4f81-8479-694e70a42439 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:38.055522 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73546db3-6982-4f81-8479-694e70a42439 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:38.186178 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1199/2374] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:39:37 2022] POST /identity/v3/users => generated 296 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:39:38.187526 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee3ed28-5f0e-44b8-893d-7082c8e92af1 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:38.198710 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee3ed28-5f0e-44b8-893d-7082c8e92af1 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:38.203521 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dbebf2-4996-4026-a5e2-5996e0e3a005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:38.204233 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbebf2-4996-4026-a5e2-5996e0e3a005 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:38.206476 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbebf2-4996-4026-a5e2-5996e0e3a005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:38.206476 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbebf2-4996-4026-a5e2-5996e0e3a005 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:38.206476 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dbebf2-4996-4026-a5e2-5996e0e3a005 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:38.207426 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dbebf2-4996-4026-a5e2-5996e0e3a005 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:38.219779 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1176/2375] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:39:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:39:38.224675 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73546db3-6982-4f81-8479-694e70a42439 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:38.236537 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73546db3-6982-4f81-8479-694e70a42439 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:38.242786 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b0167-3739-4dac-b68e-c3e9e9e1c7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:38.243616 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b0167-3739-4dac-b68e-c3e9e9e1c7f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:38.243985 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b0167-3739-4dac-b68e-c3e9e9e1c7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:38.244280 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b0167-3739-4dac-b68e-c3e9e9e1c7f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:38.245157 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334b0167-3739-4dac-b68e-c3e9e9e1c7f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:38.247020 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334b0167-3739-4dac-b68e-c3e9e9e1c7f4 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:38.258365 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1200/2376] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:39:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:39:38.262867 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dbebf2-4996-4026-a5e2-5996e0e3a005 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:38.273428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dbebf2-4996-4026-a5e2-5996e0e3a005 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:38.278365 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6998c6-1779-4511-be4c-e1d3e993cff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:38.279284 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6998c6-1779-4511-be4c-e1d3e993cff9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:38.279579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6998c6-1779-4511-be4c-e1d3e993cff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:38.279874 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6998c6-1779-4511-be4c-e1d3e993cff9 admin admin] PATH_INFO: `/v3/projects/077634babf4b433bbd3e94a3a04820e3/users/8c091a4592d94b71ae236c27717d4459/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:38.296520 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6998c6-1779-4511-be4c-e1d3e993cff9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=077634babf4b433bbd3e94a3a04820e3, user_id=8c091a4592d94b71ae236c27717d4459, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:38.299214 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6998c6-1779-4511-be4c-e1d3e993cff9 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:38.382897 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1177/2377] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:39:38 2022] PUT /identity/v3/projects/077634babf4b433bbd3e94a3a04820e3/users/8c091a4592d94b71ae236c27717d4459/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:39:38.393641 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03d7dc-e6de-4730-91c1-b55416273a42 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:38.393933 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03d7dc-e6de-4730-91c1-b55416273a42 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:38.394248 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03d7dc-e6de-4730-91c1-b55416273a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:38.429341 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-8e03d7dc-e6de-4730-91c1-b55416273a42 None None] MFA Rules not processed for user `8c091a4592d94b71ae236c27717d4459`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:39:38.430857 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-8e03d7dc-e6de-4730-91c1-b55416273a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:38.527620 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1201/2378] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:39:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:39:38.540026 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6998c6-1779-4511-be4c-e1d3e993cff9 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:38.542146 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-6c6998c6-1779-4511-be4c-e1d3e993cff9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:38.621789 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6998c6-1779-4511-be4c-e1d3e993cff9 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:38.626221 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e41c4-2954-4aba-a7cc-a89e6ab817f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:38.627076 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e41c4-2954-4aba-a7cc-a89e6ab817f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:38.629856 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e41c4-2954-4aba-a7cc-a89e6ab817f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:38.630802 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e41c4-2954-4aba-a7cc-a89e6ab817f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:38.641834 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e41c4-2954-4aba-a7cc-a89e6ab817f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:38.643314 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e41c4-2954-4aba-a7cc-a89e6ab817f9 service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:38.653375 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1178/2379] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:39:38 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:39:48.518610 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=58,keystone:INSERT=3 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:39:48.648935 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=82,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:39:56.570279 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2ea3a-f57f-457b-ab2e-d8c23dc3810e None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:56.570749 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2ea3a-f57f-457b-ab2e-d8c23dc3810e None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:56.570852 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2ea3a-f57f-457b-ab2e-d8c23dc3810e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:56.588317 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-4fb2ea3a-f57f-457b-ab2e-d8c23dc3810e None None] MFA Rules not processed for user `05e0f1ddeb224677b18cfc769dc45d47`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:39:56.589528 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-4fb2ea3a-f57f-457b-ab2e-d8c23dc3810e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:56.646569 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1202/2380] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:39:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:39:57.933090 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8b9d5-64b2-4240-93e8-c31aa98a3a92 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:57.933552 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8b9d5-64b2-4240-93e8-c31aa98a3a92 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:57.933649 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8b9d5-64b2-4240-93e8-c31aa98a3a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:57.951862 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-d1a8b9d5-64b2-4240-93e8-c31aa98a3a92 None None] MFA Rules not processed for user `cd3c04f39d774998b66e6ce920925691`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:39:57.953071 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-d1a8b9d5-64b2-4240-93e8-c31aa98a3a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:58.029950 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1179/2381] 10.222.0.25 () {58 vars in 967 bytes} [Thu Jul 7 15:39:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:39:58.049765 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb2ea3a-f57f-457b-ab2e-d8c23dc3810e None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:39:58.052091 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-4fb2ea3a-f57f-457b-ab2e-d8c23dc3810e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:39:58.139045 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb2ea3a-f57f-457b-ab2e-d8c23dc3810e None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:39:58.144210 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d473c4a9-8f17-4975-9bee-e6a8c0b0183a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b18033f849b240999d12c1885d2f7b91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:39:58.144802 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473c4a9-8f17-4975-9bee-e6a8c0b0183a service swift] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:39:58.145126 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473c4a9-8f17-4975-9bee-e6a8c0b0183a service swift] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:39:58.145426 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473c4a9-8f17-4975-9bee-e6a8c0b0183a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:39:58.154684 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d473c4a9-8f17-4975-9bee-e6a8c0b0183a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:39:58.156410 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d473c4a9-8f17-4975-9bee-e6a8c0b0183a service swift] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:39:58.165982 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1203/2382] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Jul 7 15:39:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 07 15:40:00.399388 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a8b9d5-64b2-4240-93e8-c31aa98a3a92 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:00.472554 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a8b9d5-64b2-4240-93e8-c31aa98a3a92 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:00.477049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092df894-b117-4ad7-99e6-6c4942c72678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:00.477915 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092df894-b117-4ad7-99e6-6c4942c72678 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:00.478191 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092df894-b117-4ad7-99e6-6c4942c72678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:00.478457 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092df894-b117-4ad7-99e6-6c4942c72678 admin admin] PATH_INFO: `/v3/users/8c091a4592d94b71ae236c27717d4459` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:00.479780 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092df894-b117-4ad7-99e6-6c4942c72678 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c091a4592d94b71ae236c27717d4459)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:00.481770 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092df894-b117-4ad7-99e6-6c4942c72678 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:00.751290 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-092df894-b117-4ad7-99e6-6c4942c72678 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c091a4592d94b71ae236c27717d4459'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:00.758504 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-092df894-b117-4ad7-99e6-6c4942c72678 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c091a4592d94b71ae236c27717d4459'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:00.771068 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-092df894-b117-4ad7-99e6-6c4942c72678 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c091a4592d94b71ae236c27717d4459'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:00.771806 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-092df894-b117-4ad7-99e6-6c4942c72678 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c091a4592d94b71ae236c27717d4459'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:00.855947 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-092df894-b117-4ad7-99e6-6c4942c72678 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c091a4592d94b71ae236c27717d4459'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:00.863692 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1180/2383] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:40:00 2022] DELETE /identity/v3/users/8c091a4592d94b71ae236c27717d4459 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:40:00.868679 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542c38d-6c6e-4a2c-ba39-c0bbaacf8e80 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:00.869187 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542c38d-6c6e-4a2c-ba39-c0bbaacf8e80 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:00.869288 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542c38d-6c6e-4a2c-ba39-c0bbaacf8e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:00.887885 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-c542c38d-6c6e-4a2c-ba39-c0bbaacf8e80 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:00.889119 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-c542c38d-6c6e-4a2c-ba39-c0bbaacf8e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:00.953831 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1204/2384] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:40:00.964718 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092df894-b117-4ad7-99e6-6c4942c72678 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:00.966322 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-092df894-b117-4ad7-99e6-6c4942c72678 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:00.999256 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:01.030401 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092df894-b117-4ad7-99e6-6c4942c72678 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:01.034088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac3b44-ef04-4492-8de5-d8489e33ce02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:01.034740 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac3b44-ef04-4492-8de5-d8489e33ce02 service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:01.035024 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac3b44-ef04-4492-8de5-d8489e33ce02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:01.036476 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac3b44-ef04-4492-8de5-d8489e33ce02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:01.048565 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac3b44-ef04-4492-8de5-d8489e33ce02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:01.050986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac3b44-ef04-4492-8de5-d8489e33ce02 service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:01.070852 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1181/2385] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:40:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:40:01.536469 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542c38d-6c6e-4a2c-ba39-c0bbaacf8e80 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:01.538039 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-c542c38d-6c6e-4a2c-ba39-c0bbaacf8e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:01.552600 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542c38d-6c6e-4a2c-ba39-c0bbaacf8e80 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:01.557276 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd4daa-f1cb-44c4-b897-add5da79cf9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:01.558140 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd4daa-f1cb-44c4-b897-add5da79cf9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:01.558424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd4daa-f1cb-44c4-b897-add5da79cf9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:01.558701 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd4daa-f1cb-44c4-b897-add5da79cf9d admin admin] PATH_INFO: `/v3/projects/077634babf4b433bbd3e94a3a04820e3` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:01.559928 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecd4daa-f1cb-44c4-b897-add5da79cf9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=077634babf4b433bbd3e94a3a04820e3)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:01.563371 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecd4daa-f1cb-44c4-b897-add5da79cf9d admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:01.711622 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:01.727380 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-cecd4daa-f1cb-44c4-b897-add5da79cf9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '077634babf4b433bbd3e94a3a04820e3', 'request_id': 'req-cecd4daa-f1cb-44c4-b897-add5da79cf9d'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:01.733601 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-cecd4daa-f1cb-44c4-b897-add5da79cf9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '077634babf4b433bbd3e94a3a04820e3', 'request_id': 'req-cecd4daa-f1cb-44c4-b897-add5da79cf9d'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:01.743469 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-cecd4daa-f1cb-44c4-b897-add5da79cf9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '077634babf4b433bbd3e94a3a04820e3', 'request_id': 'req-cecd4daa-f1cb-44c4-b897-add5da79cf9d'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:01.824273 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-cecd4daa-f1cb-44c4-b897-add5da79cf9d admin admin] The token cache is being invalidate because project 077634babf4b433bbd3e94a3a04820e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:40:01.824386 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-cecd4daa-f1cb-44c4-b897-add5da79cf9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:01.825267 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-cecd4daa-f1cb-44c4-b897-add5da79cf9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:01.827182 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1205/2386] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:40:01 2022] DELETE /identity/v3/projects/077634babf4b433bbd3e94a3a04820e3 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:40:01.832315 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e94fb2-20d2-478d-ac6c-0743626be007 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:01.832877 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e94fb2-20d2-478d-ac6c-0743626be007 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:01.837117 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e94fb2-20d2-478d-ac6c-0743626be007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:01.859790 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-51e94fb2-20d2-478d-ac6c-0743626be007 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:01.861284 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-51e94fb2-20d2-478d-ac6c-0743626be007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:01.929847 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1182/2387] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:40:02.003637 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c3a81-7057-48c5-9dbc-66f5b92f116a None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:02.005020 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c3a81-7057-48c5-9dbc-66f5b92f116a None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:02.005020 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c3a81-7057-48c5-9dbc-66f5b92f116a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:02.032610 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-055c3a81-7057-48c5-9dbc-66f5b92f116a None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:02.034379 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-055c3a81-7057-48c5-9dbc-66f5b92f116a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:02.054425 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1206/2388] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:40:02.058839 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e94fb2-20d2-478d-ac6c-0743626be007 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:02.081832 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e94fb2-20d2-478d-ac6c-0743626be007 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:02.093226 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e11bb35-8fc3-4db0-a08c-dabec4b426e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:02.093939 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11bb35-8fc3-4db0-a08c-dabec4b426e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:02.094219 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11bb35-8fc3-4db0-a08c-dabec4b426e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:02.094519 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11bb35-8fc3-4db0-a08c-dabec4b426e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:02.095766 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e11bb35-8fc3-4db0-a08c-dabec4b426e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:40:02.096088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e11bb35-8fc3-4db0-a08c-dabec4b426e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:02.097707 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e11bb35-8fc3-4db0-a08c-dabec4b426e8 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:02.111655 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1183/2389] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:40:02.118444 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c3a81-7057-48c5-9dbc-66f5b92f116a None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:02.135224 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c3a81-7057-48c5-9dbc-66f5b92f116a None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:02.143656 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595066e1-4f3e-442a-b0cb-bfa822d45958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:02.144820 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595066e1-4f3e-442a-b0cb-bfa822d45958 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:02.145376 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595066e1-4f3e-442a-b0cb-bfa822d45958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:02.145883 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595066e1-4f3e-442a-b0cb-bfa822d45958 admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:02.147167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595066e1-4f3e-442a-b0cb-bfa822d45958 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:02.150969 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595066e1-4f3e-442a-b0cb-bfa822d45958 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:02.262403 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1207/2390] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:40:02 2022] POST /identity/v3/projects => generated 397 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:40:02.264183 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e11bb35-8fc3-4db0-a08c-dabec4b426e8 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:02.287403 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e11bb35-8fc3-4db0-a08c-dabec4b426e8 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:02.377822 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbf150e-13d1-4554-a32b-703570de5792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:02.378506 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf150e-13d1-4554-a32b-703570de5792 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:02.378795 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf150e-13d1-4554-a32b-703570de5792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:02.379072 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf150e-13d1-4554-a32b-703570de5792 admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:02.379908 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbf150e-13d1-4554-a32b-703570de5792 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:02.382099 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbf150e-13d1-4554-a32b-703570de5792 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:02.467282 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1184/2391] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:40:02 2022] POST /identity/v3/users => generated 299 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:40:02.468507 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595066e1-4f3e-442a-b0cb-bfa822d45958 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:02.499727 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595066e1-4f3e-442a-b0cb-bfa822d45958 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:02.506488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596bb0dd-e4be-42ff-ae5f-697a6e57071e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:02.507500 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596bb0dd-e4be-42ff-ae5f-697a6e57071e admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:02.507943 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596bb0dd-e4be-42ff-ae5f-697a6e57071e admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:02.508446 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596bb0dd-e4be-42ff-ae5f-697a6e57071e admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:02.509261 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596bb0dd-e4be-42ff-ae5f-697a6e57071e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:02.511600 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596bb0dd-e4be-42ff-ae5f-697a6e57071e admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:02.536514 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1208/2392] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:40:02.540782 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbf150e-13d1-4554-a32b-703570de5792 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:02.558144 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbf150e-13d1-4554-a32b-703570de5792 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:02.564823 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa36c6-cd66-4501-9cdd-8cc117562ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:02.565807 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa36c6-cd66-4501-9cdd-8cc117562ddc admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:02.566261 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa36c6-cd66-4501-9cdd-8cc117562ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:02.569924 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa36c6-cd66-4501-9cdd-8cc117562ddc admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:02.569924 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa36c6-cd66-4501-9cdd-8cc117562ddc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:02.569924 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa36c6-cd66-4501-9cdd-8cc117562ddc admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:02.586123 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1185/2393] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:40:02.592478 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596bb0dd-e4be-42ff-ae5f-697a6e57071e admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:02.613129 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596bb0dd-e4be-42ff-ae5f-697a6e57071e admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:02.618621 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186830a8-1652-4fd9-bf9a-1fcf9a641d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:02.619918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186830a8-1652-4fd9-bf9a-1fcf9a641d2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:02.620078 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186830a8-1652-4fd9-bf9a-1fcf9a641d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:02.620430 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186830a8-1652-4fd9-bf9a-1fcf9a641d2d admin admin] PATH_INFO: `/v3/projects/bc67de22c4894bb985af3f682590d548/users/70f8e0dfc63b48ab8a9789b2d555b0e7/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:02.647511 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186830a8-1652-4fd9-bf9a-1fcf9a641d2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc67de22c4894bb985af3f682590d548, user_id=70f8e0dfc63b48ab8a9789b2d555b0e7, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:02.650273 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186830a8-1652-4fd9-bf9a-1fcf9a641d2d admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:02.790015 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1209/2394] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:40:02 2022] PUT /identity/v3/projects/bc67de22c4894bb985af3f682590d548/users/70f8e0dfc63b48ab8a9789b2d555b0e7/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:40:02.802819 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96db63-40a9-4d53-ab27-6bd08c1f16c3 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:02.802819 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96db63-40a9-4d53-ab27-6bd08c1f16c3 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:02.802819 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96db63-40a9-4d53-ab27-6bd08c1f16c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:02.830622 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-cc96db63-40a9-4d53-ab27-6bd08c1f16c3 None None] MFA Rules not processed for user `70f8e0dfc63b48ab8a9789b2d555b0e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:02.831922 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-cc96db63-40a9-4d53-ab27-6bd08c1f16c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:02.855550 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:02.908112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1186/2395] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:02 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:40:02.920627 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186830a8-1652-4fd9-bf9a-1fcf9a641d2d admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:02.924286 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-186830a8-1652-4fd9-bf9a-1fcf9a641d2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:03.002132 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186830a8-1652-4fd9-bf9a-1fcf9a641d2d admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:03.006988 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b34549-2b36-447f-9f0a-07c87a92a6a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:03.008435 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b34549-2b36-447f-9f0a-07c87a92a6a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:03.008729 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b34549-2b36-447f-9f0a-07c87a92a6a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:03.012423 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b34549-2b36-447f-9f0a-07c87a92a6a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:03.021796 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b34549-2b36-447f-9f0a-07c87a92a6a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:03.023414 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b34549-2b36-447f-9f0a-07c87a92a6a3 service cinder] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:03.033006 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1210/2396] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:40:02 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 15:40:12.142773 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc96db63-40a9-4d53-ab27-6bd08c1f16c3 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:12.208023 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc96db63-40a9-4d53-ab27-6bd08c1f16c3 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:12.212921 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06853182-8540-4c34-a63e-b3786bff0a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:12.213787 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06853182-8540-4c34-a63e-b3786bff0a3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:12.216198 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06853182-8540-4c34-a63e-b3786bff0a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:12.216583 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06853182-8540-4c34-a63e-b3786bff0a3b admin admin] PATH_INFO: `/v3/users/70f8e0dfc63b48ab8a9789b2d555b0e7` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:12.217788 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06853182-8540-4c34-a63e-b3786bff0a3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=70f8e0dfc63b48ab8a9789b2d555b0e7)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:12.219877 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06853182-8540-4c34-a63e-b3786bff0a3b admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:12.494413 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-06853182-8540-4c34-a63e-b3786bff0a3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70f8e0dfc63b48ab8a9789b2d555b0e7'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:12.511026 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-06853182-8540-4c34-a63e-b3786bff0a3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70f8e0dfc63b48ab8a9789b2d555b0e7'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:12.529977 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-06853182-8540-4c34-a63e-b3786bff0a3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70f8e0dfc63b48ab8a9789b2d555b0e7'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:12.530802 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-06853182-8540-4c34-a63e-b3786bff0a3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70f8e0dfc63b48ab8a9789b2d555b0e7'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:12.609854 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-06853182-8540-4c34-a63e-b3786bff0a3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70f8e0dfc63b48ab8a9789b2d555b0e7'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:12.616725 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1187/2397] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:40:12 2022] DELETE /identity/v3/users/70f8e0dfc63b48ab8a9789b2d555b0e7 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:40:12.621310 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f157d49-13ac-40fe-bd2b-5540a586d3a1 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:12.621797 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f157d49-13ac-40fe-bd2b-5540a586d3a1 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:12.621993 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f157d49-13ac-40fe-bd2b-5540a586d3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:12.639335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-0f157d49-13ac-40fe-bd2b-5540a586d3a1 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:12.640571 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-0f157d49-13ac-40fe-bd2b-5540a586d3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:12.694458 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:12.702912 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1211/2398] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:40:12.714372 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06853182-8540-4c34-a63e-b3786bff0a3b admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:12.715808 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-06853182-8540-4c34-a63e-b3786bff0a3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:12.763010 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:12.775170 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06853182-8540-4c34-a63e-b3786bff0a3b admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:12.778951 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e61329-6f1f-4750-b6cb-1609e20067cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:12.781012 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e61329-6f1f-4750-b6cb-1609e20067cb service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:12.781012 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e61329-6f1f-4750-b6cb-1609e20067cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:12.781012 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e61329-6f1f-4750-b6cb-1609e20067cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:12.789883 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e61329-6f1f-4750-b6cb-1609e20067cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:12.791622 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e61329-6f1f-4750-b6cb-1609e20067cb service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:12.805318 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1188/2399] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:40:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:40:13.274166 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f157d49-13ac-40fe-bd2b-5540a586d3a1 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:13.275864 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-0f157d49-13ac-40fe-bd2b-5540a586d3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:13.289449 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f157d49-13ac-40fe-bd2b-5540a586d3a1 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:13.295361 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482e355-a569-4099-a13e-9917e977e33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:13.296623 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482e355-a569-4099-a13e-9917e977e33a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:13.296967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482e355-a569-4099-a13e-9917e977e33a admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:13.297430 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482e355-a569-4099-a13e-9917e977e33a admin admin] PATH_INFO: `/v3/projects/bc67de22c4894bb985af3f682590d548` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:13.298636 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5482e355-a569-4099-a13e-9917e977e33a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc67de22c4894bb985af3f682590d548)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:13.301951 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5482e355-a569-4099-a13e-9917e977e33a admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:13.444116 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-5482e355-a569-4099-a13e-9917e977e33a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc67de22c4894bb985af3f682590d548', 'request_id': 'req-5482e355-a569-4099-a13e-9917e977e33a'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:13.449839 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-5482e355-a569-4099-a13e-9917e977e33a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc67de22c4894bb985af3f682590d548', 'request_id': 'req-5482e355-a569-4099-a13e-9917e977e33a'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:13.456202 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-5482e355-a569-4099-a13e-9917e977e33a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc67de22c4894bb985af3f682590d548', 'request_id': 'req-5482e355-a569-4099-a13e-9917e977e33a'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:13.546344 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-5482e355-a569-4099-a13e-9917e977e33a admin admin] The token cache is being invalidate because project bc67de22c4894bb985af3f682590d548 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:40:13.546895 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-5482e355-a569-4099-a13e-9917e977e33a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:13.550100 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-5482e355-a569-4099-a13e-9917e977e33a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:13.562334 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1212/2400] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:40:13 2022] DELETE /identity/v3/projects/bc67de22c4894bb985af3f682590d548 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:40:23.191272 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=7 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:23.542944 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=28,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:26.018732 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e61329-6f1f-4750-b6cb-1609e20067cb service neutron] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:26.026292 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-c2e61329-6f1f-4750-b6cb-1609e20067cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:26.032355 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e61329-6f1f-4750-b6cb-1609e20067cb service neutron] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:26.093299 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea820da-bf19-419c-bec9-aecd947d2009 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:26.093983 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea820da-bf19-419c-bec9-aecd947d2009 service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:26.094262 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea820da-bf19-419c-bec9-aecd947d2009 service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:26.094536 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea820da-bf19-419c-bec9-aecd947d2009 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:26.096292 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-2ea820da-bf19-419c-bec9-aecd947d2009 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:26.160908 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea820da-bf19-419c-bec9-aecd947d2009 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:26.162255 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea820da-bf19-419c-bec9-aecd947d2009 service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:26.172561 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1189/2401] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:40:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:40:27.448090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482e355-a569-4099-a13e-9917e977e33a admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:27.449692 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-5482e355-a569-4099-a13e-9917e977e33a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:27.466071 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482e355-a569-4099-a13e-9917e977e33a admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:27.470348 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849a8e54-2e92-4cea-9959-a385c84bfc01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:27.471384 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a8e54-2e92-4cea-9959-a385c84bfc01 service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:27.471873 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a8e54-2e92-4cea-9959-a385c84bfc01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:27.472380 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a8e54-2e92-4cea-9959-a385c84bfc01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:27.475133 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-849a8e54-2e92-4cea-9959-a385c84bfc01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:27.512582 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849a8e54-2e92-4cea-9959-a385c84bfc01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:27.515289 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849a8e54-2e92-4cea-9959-a385c84bfc01 service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:27.546204 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1213/2402] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:40:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:40:30.588262 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea820da-bf19-419c-bec9-aecd947d2009 service neutron] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:30.590010 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-2ea820da-bf19-419c-bec9-aecd947d2009 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:30.605087 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea820da-bf19-419c-bec9-aecd947d2009 service neutron] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:30.610147 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8102b6-2459-4a81-bddf-24b8170219e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:30.611446 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8102b6-2459-4a81-bddf-24b8170219e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:30.611862 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8102b6-2459-4a81-bddf-24b8170219e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:30.612230 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8102b6-2459-4a81-bddf-24b8170219e2 admin admin] PATH_INFO: `/v3/users/b2e09539caff4d4698cf29c1ea22d365` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:30.614129 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8102b6-2459-4a81-bddf-24b8170219e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2e09539caff4d4698cf29c1ea22d365)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:30.616386 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8102b6-2459-4a81-bddf-24b8170219e2 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:30.817952 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-dd8102b6-2459-4a81-bddf-24b8170219e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2e09539caff4d4698cf29c1ea22d365'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:30.825068 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-dd8102b6-2459-4a81-bddf-24b8170219e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2e09539caff4d4698cf29c1ea22d365'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:30.837405 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-dd8102b6-2459-4a81-bddf-24b8170219e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2e09539caff4d4698cf29c1ea22d365'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:30.838103 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-dd8102b6-2459-4a81-bddf-24b8170219e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2e09539caff4d4698cf29c1ea22d365'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:30.916949 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-dd8102b6-2459-4a81-bddf-24b8170219e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2e09539caff4d4698cf29c1ea22d365'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:30.923340 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1190/2403] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:40:30 2022] DELETE /identity/v3/users/b2e09539caff4d4698cf29c1ea22d365 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:40:31.182951 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849a8e54-2e92-4cea-9959-a385c84bfc01 service neutron] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:31.196419 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849a8e54-2e92-4cea-9959-a385c84bfc01 service neutron] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:31.264339 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c3783-922b-423a-93a3-a4a01954950c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:31.265192 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c3783-922b-423a-93a3-a4a01954950c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:31.265478 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c3783-922b-423a-93a3-a4a01954950c admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:31.265752 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c3783-922b-423a-93a3-a4a01954950c admin admin] PATH_INFO: `/v3/projects/66b37250ea6d4c4eb242c27ce9819dfc` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:31.266931 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152c3783-922b-423a-93a3-a4a01954950c admin admin] RBAC: Authorizing `identity:delete_project(project_id=66b37250ea6d4c4eb242c27ce9819dfc)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:31.268897 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152c3783-922b-423a-93a3-a4a01954950c admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:31.389376 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-152c3783-922b-423a-93a3-a4a01954950c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66b37250ea6d4c4eb242c27ce9819dfc', 'request_id': 'req-152c3783-922b-423a-93a3-a4a01954950c'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:31.401575 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-152c3783-922b-423a-93a3-a4a01954950c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66b37250ea6d4c4eb242c27ce9819dfc', 'request_id': 'req-152c3783-922b-423a-93a3-a4a01954950c'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:31.409622 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-152c3783-922b-423a-93a3-a4a01954950c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66b37250ea6d4c4eb242c27ce9819dfc', 'request_id': 'req-152c3783-922b-423a-93a3-a4a01954950c'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:31.547163 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-152c3783-922b-423a-93a3-a4a01954950c admin admin] The token cache is being invalidate because project 66b37250ea6d4c4eb242c27ce9819dfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:40:31.547513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-152c3783-922b-423a-93a3-a4a01954950c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:31.555046 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-152c3783-922b-423a-93a3-a4a01954950c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:31.560354 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1214/2404] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:40:31 2022] DELETE /identity/v3/projects/66b37250ea6d4c4eb242c27ce9819dfc => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:40:31.562109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1263b8-8ee7-4d88-8a0e-5703cded6f0c None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:31.562468 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1263b8-8ee7-4d88-8a0e-5703cded6f0c None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:31.562790 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1263b8-8ee7-4d88-8a0e-5703cded6f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:31.585043 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-ae1263b8-8ee7-4d88-8a0e-5703cded6f0c None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:31.586648 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-ae1263b8-8ee7-4d88-8a0e-5703cded6f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:31.606626 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:31.651602 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1191/2405] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:40:31.659881 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941a0b5-af59-4699-9140-0ae927bd90de None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:31.660171 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941a0b5-af59-4699-9140-0ae927bd90de None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:31.660456 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941a0b5-af59-4699-9140-0ae927bd90de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:31.681757 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-c941a0b5-af59-4699-9140-0ae927bd90de None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:31.682978 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-c941a0b5-af59-4699-9140-0ae927bd90de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:31.691644 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1215/2406] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:40:31.696098 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1263b8-8ee7-4d88-8a0e-5703cded6f0c None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:31.707618 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1263b8-8ee7-4d88-8a0e-5703cded6f0c None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:31.713974 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852e247-2510-47e6-bbea-0f461ee65b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:31.718653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852e247-2510-47e6-bbea-0f461ee65b1a admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:31.718653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852e247-2510-47e6-bbea-0f461ee65b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:31.718653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852e247-2510-47e6-bbea-0f461ee65b1a admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:31.718653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0852e247-2510-47e6-bbea-0f461ee65b1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:40:31.718653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0852e247-2510-47e6-bbea-0f461ee65b1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:31.718653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0852e247-2510-47e6-bbea-0f461ee65b1a admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:31.741496 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1192/2407] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:40:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:40:31.745082 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c941a0b5-af59-4699-9140-0ae927bd90de None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:31.755595 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c941a0b5-af59-4699-9140-0ae927bd90de None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:31.762488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb725b9f-be4a-46ee-a759-35b5bb875d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:31.762488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb725b9f-be4a-46ee-a759-35b5bb875d49 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:31.762488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb725b9f-be4a-46ee-a759-35b5bb875d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:31.762488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb725b9f-be4a-46ee-a759-35b5bb875d49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:31.762900 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb725b9f-be4a-46ee-a759-35b5bb875d49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:31.764663 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb725b9f-be4a-46ee-a759-35b5bb875d49 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:31.848771 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1216/2408] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:40:31 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:40:31.854752 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852e247-2510-47e6-bbea-0f461ee65b1a admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:31.860321 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852e247-2510-47e6-bbea-0f461ee65b1a admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:31.921445 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc5393-bf79-44d2-89c7-6cf7e2d43833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:31.922118 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc5393-bf79-44d2-89c7-6cf7e2d43833 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:31.922399 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc5393-bf79-44d2-89c7-6cf7e2d43833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:31.922679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc5393-bf79-44d2-89c7-6cf7e2d43833 admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:31.923431 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc5393-bf79-44d2-89c7-6cf7e2d43833 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:31.925895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc5393-bf79-44d2-89c7-6cf7e2d43833 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:32.017702 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb725b9f-be4a-46ee-a759-35b5bb875d49 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:32.019825 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-fb725b9f-be4a-46ee-a759-35b5bb875d49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:32.028048 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1193/2409] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:40:31 2022] POST /identity/v3/users => generated 287 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:40:32.033854 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc5393-bf79-44d2-89c7-6cf7e2d43833 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:32.034460 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb725b9f-be4a-46ee-a759-35b5bb875d49 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:32.039222 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f327d927-0ca0-4995-af00-f106bfd5a5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:32.040124 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327d927-0ca0-4995-af00-f106bfd5a5d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:32.040406 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327d927-0ca0-4995-af00-f106bfd5a5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:32.043828 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327d927-0ca0-4995-af00-f106bfd5a5d0 admin admin] PATH_INFO: `/v3/users/b5dec80a83aa463b99bd298ab09dfbdb` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:32.043828 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f327d927-0ca0-4995-af00-f106bfd5a5d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5dec80a83aa463b99bd298ab09dfbdb)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:32.044255 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc5393-bf79-44d2-89c7-6cf7e2d43833 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:32.044425 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f327d927-0ca0-4995-af00-f106bfd5a5d0 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:32.048878 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7566ef-7ea6-4a08-b6b2-7d1777db7224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:32.049530 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7566ef-7ea6-4a08-b6b2-7d1777db7224 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:32.049811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7566ef-7ea6-4a08-b6b2-7d1777db7224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:32.050092 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7566ef-7ea6-4a08-b6b2-7d1777db7224 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:32.050894 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7566ef-7ea6-4a08-b6b2-7d1777db7224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:32.052997 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7566ef-7ea6-4a08-b6b2-7d1777db7224 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:32.063263 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1194/2410] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:40:32.066979 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7566ef-7ea6-4a08-b6b2-7d1777db7224 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:32.077241 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7566ef-7ea6-4a08-b6b2-7d1777db7224 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:32.081795 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e8c0a-178d-4008-8065-77accfba7d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:32.082446 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e8c0a-178d-4008-8065-77accfba7d1f admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:32.082759 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e8c0a-178d-4008-8065-77accfba7d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:32.083057 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e8c0a-178d-4008-8065-77accfba7d1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:32.083809 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e8c0a-178d-4008-8065-77accfba7d1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:32.085353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e8c0a-178d-4008-8065-77accfba7d1f admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:32.095132 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1195/2411] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:40:32.099537 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e8c0a-178d-4008-8065-77accfba7d1f admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:32.108771 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e8c0a-178d-4008-8065-77accfba7d1f admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:32.113601 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ea41d-0ee1-4030-aa4c-bb68d6acaa3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:32.115056 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ea41d-0ee1-4030-aa4c-bb68d6acaa3a admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:32.115339 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ea41d-0ee1-4030-aa4c-bb68d6acaa3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:32.115670 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ea41d-0ee1-4030-aa4c-bb68d6acaa3a admin admin] PATH_INFO: `/v3/projects/0cc447a829d940d78ebc638f063a3408/users/f86cfc5a1bd742b8b33d513268d9dc0f/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:32.130000 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ea41d-0ee1-4030-aa4c-bb68d6acaa3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cc447a829d940d78ebc638f063a3408, user_id=f86cfc5a1bd742b8b33d513268d9dc0f, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:32.132640 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ea41d-0ee1-4030-aa4c-bb68d6acaa3a admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:32.197983 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1196/2412] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:40:32 2022] PUT /identity/v3/projects/0cc447a829d940d78ebc638f063a3408/users/f86cfc5a1bd742b8b33d513268d9dc0f/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:40:32.208076 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef669c-d030-4ca6-9ce8-e224e9e7c42e None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:32.208364 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef669c-d030-4ca6-9ce8-e224e9e7c42e None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:32.208641 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef669c-d030-4ca6-9ce8-e224e9e7c42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:32.222360 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:32.225379 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-f327d927-0ca0-4995-af00-f106bfd5a5d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5dec80a83aa463b99bd298ab09dfbdb'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:32.226059 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-f327d927-0ca0-4995-af00-f106bfd5a5d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5dec80a83aa463b99bd298ab09dfbdb'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:32.233445 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-f327d927-0ca0-4995-af00-f106bfd5a5d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5dec80a83aa463b99bd298ab09dfbdb'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:32.234014 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-f327d927-0ca0-4995-af00-f106bfd5a5d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5dec80a83aa463b99bd298ab09dfbdb'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:32.236149 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-b1ef669c-d030-4ca6-9ce8-e224e9e7c42e None None] MFA Rules not processed for user `f86cfc5a1bd742b8b33d513268d9dc0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:32.237406 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-b1ef669c-d030-4ca6-9ce8-e224e9e7c42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:32.246597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-f327d927-0ca0-4995-af00-f106bfd5a5d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5dec80a83aa463b99bd298ab09dfbdb'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:32.278691 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:32.303029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1197/2413] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:40:32.309513 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1217/2414] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:40:32 2022] DELETE /identity/v3/users/b5dec80a83aa463b99bd298ab09dfbdb => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:40:32.316617 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef669c-d030-4ca6-9ce8-e224e9e7c42e None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:32.318173 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-b1ef669c-d030-4ca6-9ce8-e224e9e7c42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:32.393810 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef669c-d030-4ca6-9ce8-e224e9e7c42e None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:32.467960 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dad322-1413-40d1-8d92-f2b66485d8f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:32.468827 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dad322-1413-40d1-8d92-f2b66485d8f2 service nova] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:32.469101 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dad322-1413-40d1-8d92-f2b66485d8f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:32.469415 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dad322-1413-40d1-8d92-f2b66485d8f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:32.527172 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dad322-1413-40d1-8d92-f2b66485d8f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:32.528781 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dad322-1413-40d1-8d92-f2b66485d8f2 service nova] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:32.538582 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1198/2415] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:40:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:40:32.549999 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f327d927-0ca0-4995-af00-f106bfd5a5d0 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:32.552678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-f327d927-0ca0-4995-af00-f106bfd5a5d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:32.623478 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f327d927-0ca0-4995-af00-f106bfd5a5d0 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:32.628743 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7731b8f-d1af-4302-96c6-41b07bc13fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:32.629603 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7731b8f-d1af-4302-96c6-41b07bc13fc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:32.629920 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7731b8f-d1af-4302-96c6-41b07bc13fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:32.630199 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7731b8f-d1af-4302-96c6-41b07bc13fc4 admin admin] PATH_INFO: `/v3/projects/66a98614df914018b7ecfb68afb735c0` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:32.631388 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7731b8f-d1af-4302-96c6-41b07bc13fc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66a98614df914018b7ecfb68afb735c0)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:32.633345 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7731b8f-d1af-4302-96c6-41b07bc13fc4 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:32.756836 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-d7731b8f-d1af-4302-96c6-41b07bc13fc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66a98614df914018b7ecfb68afb735c0', 'request_id': 'req-d7731b8f-d1af-4302-96c6-41b07bc13fc4'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:32.762371 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-d7731b8f-d1af-4302-96c6-41b07bc13fc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66a98614df914018b7ecfb68afb735c0', 'request_id': 'req-d7731b8f-d1af-4302-96c6-41b07bc13fc4'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:32.769212 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-d7731b8f-d1af-4302-96c6-41b07bc13fc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66a98614df914018b7ecfb68afb735c0', 'request_id': 'req-d7731b8f-d1af-4302-96c6-41b07bc13fc4'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:32.843521 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-d7731b8f-d1af-4302-96c6-41b07bc13fc4 admin admin] The token cache is being invalidate because project 66a98614df914018b7ecfb68afb735c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:40:32.843846 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-d7731b8f-d1af-4302-96c6-41b07bc13fc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:32.844529 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-d7731b8f-d1af-4302-96c6-41b07bc13fc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:32.846989 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1218/2416] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:40:32 2022] DELETE /identity/v3/projects/66a98614df914018b7ecfb68afb735c0 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:40:32.852435 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af744b-a29b-4c8d-aba9-6219996d7ce3 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:32.852952 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af744b-a29b-4c8d-aba9-6219996d7ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:32.853051 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af744b-a29b-4c8d-aba9-6219996d7ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:32.872464 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-25af744b-a29b-4c8d-aba9-6219996d7ce3 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:32.874151 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-25af744b-a29b-4c8d-aba9-6219996d7ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:32.887021 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=100 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:32.935053 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1199/2417] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:40:32.943628 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23784ad8-9030-47a0-b1d8-8855c74826ca None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:32.943922 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23784ad8-9030-47a0-b1d8-8855c74826ca None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:32.944203 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23784ad8-9030-47a0-b1d8-8855c74826ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:32.966366 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-23784ad8-9030-47a0-b1d8-8855c74826ca None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:32.967598 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-23784ad8-9030-47a0-b1d8-8855c74826ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:32.976517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1219/2418] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:40:32.980628 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25af744b-a29b-4c8d-aba9-6219996d7ce3 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:32.990454 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25af744b-a29b-4c8d-aba9-6219996d7ce3 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:32.995130 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e10669-abad-4c87-8c07-5f01c4958650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:32.995810 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e10669-abad-4c87-8c07-5f01c4958650 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:32.996091 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e10669-abad-4c87-8c07-5f01c4958650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:32.996364 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e10669-abad-4c87-8c07-5f01c4958650 admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:32.997210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e10669-abad-4c87-8c07-5f01c4958650 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:40:32.997513 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e10669-abad-4c87-8c07-5f01c4958650 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:32.999415 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e10669-abad-4c87-8c07-5f01c4958650 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:33.019749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1200/2419] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:40:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:40:33.019878 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23784ad8-9030-47a0-b1d8-8855c74826ca None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:33.028015 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23784ad8-9030-47a0-b1d8-8855c74826ca None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:33.032767 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddfe721-e62f-4f2a-bf1a-85d830b4ba0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:33.033441 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfe721-e62f-4f2a-bf1a-85d830b4ba0c admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:33.033707 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfe721-e62f-4f2a-bf1a-85d830b4ba0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:33.034013 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfe721-e62f-4f2a-bf1a-85d830b4ba0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:33.034714 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddfe721-e62f-4f2a-bf1a-85d830b4ba0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:33.037210 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddfe721-e62f-4f2a-bf1a-85d830b4ba0c admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:33.103950 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1220/2420] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:40:33 2022] POST /identity/v3/projects => generated 371 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:40:33.113060 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e10669-abad-4c87-8c07-5f01c4958650 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:33.130161 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e10669-abad-4c87-8c07-5f01c4958650 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:33.188367 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a8072-eb3c-4933-ba26-e9c5b386b1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:33.189641 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a8072-eb3c-4933-ba26-e9c5b386b1ab admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:33.189641 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a8072-eb3c-4933-ba26-e9c5b386b1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:33.190105 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a8072-eb3c-4933-ba26-e9c5b386b1ab admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:33.190911 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00a8072-eb3c-4933-ba26-e9c5b386b1ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:33.192990 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00a8072-eb3c-4933-ba26-e9c5b386b1ab admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:33.293831 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1201/2421] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:40:33 2022] POST /identity/v3/users => generated 286 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:40:33.298046 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddfe721-e62f-4f2a-bf1a-85d830b4ba0c admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:33.308517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddfe721-e62f-4f2a-bf1a-85d830b4ba0c admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:33.316975 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e674ee8c-50ec-4a31-927d-3a55e34c6f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:33.316975 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674ee8c-50ec-4a31-927d-3a55e34c6f8f admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:33.316975 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674ee8c-50ec-4a31-927d-3a55e34c6f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:33.316975 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674ee8c-50ec-4a31-927d-3a55e34c6f8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:33.317782 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e674ee8c-50ec-4a31-927d-3a55e34c6f8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:33.317782 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e674ee8c-50ec-4a31-927d-3a55e34c6f8f admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:33.328125 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1221/2422] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:40:33.331951 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a8072-eb3c-4933-ba26-e9c5b386b1ab admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:33.342805 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a8072-eb3c-4933-ba26-e9c5b386b1ab admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:33.347554 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4158a14-32af-4f50-a25f-c16dfcd2da92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:33.348237 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4158a14-32af-4f50-a25f-c16dfcd2da92 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:33.348521 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4158a14-32af-4f50-a25f-c16dfcd2da92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:33.349449 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4158a14-32af-4f50-a25f-c16dfcd2da92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:33.350186 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4158a14-32af-4f50-a25f-c16dfcd2da92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:33.351817 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4158a14-32af-4f50-a25f-c16dfcd2da92 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:33.361974 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1202/2423] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:40:33.366034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e674ee8c-50ec-4a31-927d-3a55e34c6f8f admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:33.377918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e674ee8c-50ec-4a31-927d-3a55e34c6f8f admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:33.391692 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af0440d-9bae-47cf-a6dc-afa0f5ac7d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:33.397014 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0440d-9bae-47cf-a6dc-afa0f5ac7d1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:33.397014 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0440d-9bae-47cf-a6dc-afa0f5ac7d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:33.397014 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0440d-9bae-47cf-a6dc-afa0f5ac7d1c admin admin] PATH_INFO: `/v3/projects/579253bda38141ba973709c0a2412017/users/98e8c461c4034b58b072469927af7d1c/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:33.405134 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:33.412914 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af0440d-9bae-47cf-a6dc-afa0f5ac7d1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=579253bda38141ba973709c0a2412017, user_id=98e8c461c4034b58b072469927af7d1c, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:33.415663 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af0440d-9bae-47cf-a6dc-afa0f5ac7d1c admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:33.484947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1222/2424] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:40:33 2022] PUT /identity/v3/projects/579253bda38141ba973709c0a2412017/users/98e8c461c4034b58b072469927af7d1c/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:40:33.497579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ebe69d-052f-49eb-95ba-37c05d6f1c3a None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:33.497868 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ebe69d-052f-49eb-95ba-37c05d6f1c3a None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:33.498143 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ebe69d-052f-49eb-95ba-37c05d6f1c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:33.528428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-e2ebe69d-052f-49eb-95ba-37c05d6f1c3a None None] MFA Rules not processed for user `98e8c461c4034b58b072469927af7d1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:33.529660 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-e2ebe69d-052f-49eb-95ba-37c05d6f1c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:33.602649 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1203/2425] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:40:33.607211 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af0440d-9bae-47cf-a6dc-afa0f5ac7d1c admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:33.674258 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af0440d-9bae-47cf-a6dc-afa0f5ac7d1c admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:33.681986 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f8bebc-98f9-4170-ae75-3206be33a6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:33.681986 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8bebc-98f9-4170-ae75-3206be33a6f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:33.681986 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8bebc-98f9-4170-ae75-3206be33a6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:33.681986 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8bebc-98f9-4170-ae75-3206be33a6f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:33.682403 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f8bebc-98f9-4170-ae75-3206be33a6f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:33.683453 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f8bebc-98f9-4170-ae75-3206be33a6f2 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:33.777506 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1223/2426] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:40:33 2022] POST /identity/v3/projects => generated 373 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:40:33.781658 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ebe69d-052f-49eb-95ba-37c05d6f1c3a None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:33.792899 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ebe69d-052f-49eb-95ba-37c05d6f1c3a None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:33.801713 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:33.849605 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728afd48-64d6-4186-8ddf-412bd07f87a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:33.850185 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728afd48-64d6-4186-8ddf-412bd07f87a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:33.850464 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728afd48-64d6-4186-8ddf-412bd07f87a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:33.850731 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728afd48-64d6-4186-8ddf-412bd07f87a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:33.851530 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728afd48-64d6-4186-8ddf-412bd07f87a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:33.853474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728afd48-64d6-4186-8ddf-412bd07f87a6 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:33.980992 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1204/2427] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:40:33 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:40:33.984851 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f8bebc-98f9-4170-ae75-3206be33a6f2 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:33.995770 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f8bebc-98f9-4170-ae75-3206be33a6f2 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:34.000334 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e128762-6197-4867-8597-61132b54da1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:34.000989 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e128762-6197-4867-8597-61132b54da1b admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:34.001272 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e128762-6197-4867-8597-61132b54da1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:34.001549 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e128762-6197-4867-8597-61132b54da1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:34.002275 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e128762-6197-4867-8597-61132b54da1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:34.003908 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e128762-6197-4867-8597-61132b54da1b admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:34.017639 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1224/2428] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:40:34.021613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728afd48-64d6-4186-8ddf-412bd07f87a6 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:34.031621 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728afd48-64d6-4186-8ddf-412bd07f87a6 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:34.039001 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0e0e5-26b2-4144-9108-1c81481680f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:34.040528 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0e0e5-26b2-4144-9108-1c81481680f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:34.041316 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0e0e5-26b2-4144-9108-1c81481680f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:34.041741 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0e0e5-26b2-4144-9108-1c81481680f0 admin admin] PATH_INFO: `/v3/projects/2dfe442d438f49549480ff42ca075de6/users/c544fc4690a04dad9108f27e6afc1fcc/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:34.061562 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b0e0e5-26b2-4144-9108-1c81481680f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dfe442d438f49549480ff42ca075de6, user_id=c544fc4690a04dad9108f27e6afc1fcc, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:34.064289 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b0e0e5-26b2-4144-9108-1c81481680f0 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:34.152495 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1205/2429] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:40:34 2022] PUT /identity/v3/projects/2dfe442d438f49549480ff42ca075de6/users/c544fc4690a04dad9108f27e6afc1fcc/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:40:34.161629 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e128762-6197-4867-8597-61132b54da1b admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:34.172190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e128762-6197-4867-8597-61132b54da1b admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:34.227086 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd599c13-ca4e-4b1f-bfce-b264871e7308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:34.227784 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd599c13-ca4e-4b1f-bfce-b264871e7308 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:34.228071 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd599c13-ca4e-4b1f-bfce-b264871e7308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:34.228345 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd599c13-ca4e-4b1f-bfce-b264871e7308 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:34.229065 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd599c13-ca4e-4b1f-bfce-b264871e7308 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:34.230659 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd599c13-ca4e-4b1f-bfce-b264871e7308 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:34.240850 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1225/2430] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:40:34 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:40:34.244948 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0e0e5-26b2-4144-9108-1c81481680f0 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:34.255794 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0e0e5-26b2-4144-9108-1c81481680f0 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:34.260403 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0053e04b-2b3d-4e54-b143-eb789966bc2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:34.261266 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0053e04b-2b3d-4e54-b143-eb789966bc2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:34.261551 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0053e04b-2b3d-4e54-b143-eb789966bc2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:34.261830 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0053e04b-2b3d-4e54-b143-eb789966bc2e admin admin] PATH_INFO: `/v3/projects/2dfe442d438f49549480ff42ca075de6/users/c544fc4690a04dad9108f27e6afc1fcc/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:34.263810 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0053e04b-2b3d-4e54-b143-eb789966bc2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dfe442d438f49549480ff42ca075de6, user_id=c544fc4690a04dad9108f27e6afc1fcc, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:34.266450 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0053e04b-2b3d-4e54-b143-eb789966bc2e admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:34.276443 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1206/2431] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:40:34 2022] PUT /identity/v3/projects/2dfe442d438f49549480ff42ca075de6/users/c544fc4690a04dad9108f27e6afc1fcc/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:40:34.287455 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004fc25-9cfb-4db8-9838-b5100c723e36 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:34.287842 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004fc25-9cfb-4db8-9838-b5100c723e36 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:34.288124 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004fc25-9cfb-4db8-9838-b5100c723e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:34.315771 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-2004fc25-9cfb-4db8-9838-b5100c723e36 None None] MFA Rules not processed for user `c544fc4690a04dad9108f27e6afc1fcc`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:34.316977 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-2004fc25-9cfb-4db8-9838-b5100c723e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:34.362708 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:34.387080 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1226/2432] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:34 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:40:34.399544 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0053e04b-2b3d-4e54-b143-eb789966bc2e admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:34.401359 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-0053e04b-2b3d-4e54-b143-eb789966bc2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:34.471892 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0053e04b-2b3d-4e54-b143-eb789966bc2e admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:34.471892 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec3a1f7-0c38-4cb0-a51d-4099c8f93265 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:34.471892 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3a1f7-0c38-4cb0-a51d-4099c8f93265 service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:34.471892 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3a1f7-0c38-4cb0-a51d-4099c8f93265 service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:34.472247 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3a1f7-0c38-4cb0-a51d-4099c8f93265 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:34.480424 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec3a1f7-0c38-4cb0-a51d-4099c8f93265 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:34.481842 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec3a1f7-0c38-4cb0-a51d-4099c8f93265 service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:34.492513 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1207/2433] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:40:34 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:40:35.414987 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2004fc25-9cfb-4db8-9838-b5100c723e36 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:35.478814 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2004fc25-9cfb-4db8-9838-b5100c723e36 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:35.483512 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9be1c4-4a65-4146-811f-b52ea282d252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:35.484456 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9be1c4-4a65-4146-811f-b52ea282d252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:35.484742 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9be1c4-4a65-4146-811f-b52ea282d252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:35.485018 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9be1c4-4a65-4146-811f-b52ea282d252 admin admin] PATH_INFO: `/v3/users/98e8c461c4034b58b072469927af7d1c` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:35.491531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9be1c4-4a65-4146-811f-b52ea282d252 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98e8c461c4034b58b072469927af7d1c)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:35.491531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9be1c4-4a65-4146-811f-b52ea282d252 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:35.691218 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-dd9be1c4-4a65-4146-811f-b52ea282d252 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98e8c461c4034b58b072469927af7d1c'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:35.691966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-dd9be1c4-4a65-4146-811f-b52ea282d252 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98e8c461c4034b58b072469927af7d1c'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:35.700234 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-dd9be1c4-4a65-4146-811f-b52ea282d252 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98e8c461c4034b58b072469927af7d1c'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:35.700921 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-dd9be1c4-4a65-4146-811f-b52ea282d252 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98e8c461c4034b58b072469927af7d1c'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:35.731270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-dd9be1c4-4a65-4146-811f-b52ea282d252 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98e8c461c4034b58b072469927af7d1c'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:35.767163 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec3a1f7-0c38-4cb0-a51d-4099c8f93265 service cinder] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:35.767163 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-aec3a1f7-0c38-4cb0-a51d-4099c8f93265 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:35.785916 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec3a1f7-0c38-4cb0-a51d-4099c8f93265 service cinder] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:35.791004 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b72893-5d78-40b9-bcbe-6ddd9bf61be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:35.791995 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b72893-5d78-40b9-bcbe-6ddd9bf61be8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:35.792284 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b72893-5d78-40b9-bcbe-6ddd9bf61be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:35.792539 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b72893-5d78-40b9-bcbe-6ddd9bf61be8 admin admin] PATH_INFO: `/v3/users/f86cfc5a1bd742b8b33d513268d9dc0f` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:35.794040 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b72893-5d78-40b9-bcbe-6ddd9bf61be8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f86cfc5a1bd742b8b33d513268d9dc0f)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:35.796285 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b72893-5d78-40b9-bcbe-6ddd9bf61be8 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:35.819651 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1227/2434] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:40:35 2022] DELETE /identity/v3/users/98e8c461c4034b58b072469927af7d1c => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:40:35.825091 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff956cf3-2854-4d4f-85af-e290674d54f6 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:35.825475 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff956cf3-2854-4d4f-85af-e290674d54f6 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:35.825880 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff956cf3-2854-4d4f-85af-e290674d54f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:35.848219 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-ff956cf3-2854-4d4f-85af-e290674d54f6 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:35.849526 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-ff956cf3-2854-4d4f-85af-e290674d54f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:35.962340 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1228/2435] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:40:35.972685 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff956cf3-2854-4d4f-85af-e290674d54f6 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:35.975060 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-ff956cf3-2854-4d4f-85af-e290674d54f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:35.983086 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:35.999649 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-a9b72893-5d78-40b9-bcbe-6ddd9bf61be8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f86cfc5a1bd742b8b33d513268d9dc0f'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:36.015159 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-a9b72893-5d78-40b9-bcbe-6ddd9bf61be8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f86cfc5a1bd742b8b33d513268d9dc0f'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:36.022030 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:36.031866 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-a9b72893-5d78-40b9-bcbe-6ddd9bf61be8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f86cfc5a1bd742b8b33d513268d9dc0f'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:36.032718 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-a9b72893-5d78-40b9-bcbe-6ddd9bf61be8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f86cfc5a1bd742b8b33d513268d9dc0f'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:36.061798 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff956cf3-2854-4d4f-85af-e290674d54f6 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:36.066908 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c270a-6849-4d0b-91f1-1f0883b212ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:36.068867 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c270a-6849-4d0b-91f1-1f0883b212ae service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:36.069426 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c270a-6849-4d0b-91f1-1f0883b212ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:36.070094 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c270a-6849-4d0b-91f1-1f0883b212ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:36.084559 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212c270a-6849-4d0b-91f1-1f0883b212ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:36.087411 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212c270a-6849-4d0b-91f1-1f0883b212ae service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:36.109575 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1229/2436] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:40:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:40:36.123323 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-a9b72893-5d78-40b9-bcbe-6ddd9bf61be8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f86cfc5a1bd742b8b33d513268d9dc0f'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:36.133983 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1208/2437] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:40:35 2022] DELETE /identity/v3/users/f86cfc5a1bd742b8b33d513268d9dc0f => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:40:36.137084 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d39e70-cf58-4f53-aaad-65ab7b39baa1 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:36.137321 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d39e70-cf58-4f53-aaad-65ab7b39baa1 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:36.137594 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d39e70-cf58-4f53-aaad-65ab7b39baa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:36.164377 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-c7d39e70-cf58-4f53-aaad-65ab7b39baa1 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:36.165619 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-c7d39e70-cf58-4f53-aaad-65ab7b39baa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:36.233183 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1230/2438] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:40:36.243296 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b72893-5d78-40b9-bcbe-6ddd9bf61be8 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:36.248772 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-a9b72893-5d78-40b9-bcbe-6ddd9bf61be8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:36.303605 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b72893-5d78-40b9-bcbe-6ddd9bf61be8 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:36.307552 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c27f85-3772-438f-ae5d-8a237e2bcfb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:36.308242 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c27f85-3772-438f-ae5d-8a237e2bcfb5 service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:36.308535 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c27f85-3772-438f-ae5d-8a237e2bcfb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:36.308817 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c27f85-3772-438f-ae5d-8a237e2bcfb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:36.320068 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c27f85-3772-438f-ae5d-8a237e2bcfb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:36.321792 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c27f85-3772-438f-ae5d-8a237e2bcfb5 service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:36.332841 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1209/2439] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:40:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:40:36.648635 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d39e70-cf58-4f53-aaad-65ab7b39baa1 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:36.650142 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-c7d39e70-cf58-4f53-aaad-65ab7b39baa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:36.665945 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d39e70-cf58-4f53-aaad-65ab7b39baa1 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:36.696660 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0416e29b-c8e4-436b-895c-48a609d85a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:36.698115 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0416e29b-c8e4-436b-895c-48a609d85a00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:36.698778 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0416e29b-c8e4-436b-895c-48a609d85a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:36.699382 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0416e29b-c8e4-436b-895c-48a609d85a00 admin admin] PATH_INFO: `/v3/projects/579253bda38141ba973709c0a2412017` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:36.700959 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0416e29b-c8e4-436b-895c-48a609d85a00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=579253bda38141ba973709c0a2412017)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:36.702894 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0416e29b-c8e4-436b-895c-48a609d85a00 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:36.806569 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c27f85-3772-438f-ae5d-8a237e2bcfb5 service neutron] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:36.808130 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-05c27f85-3772-438f-ae5d-8a237e2bcfb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:36.828319 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c27f85-3772-438f-ae5d-8a237e2bcfb5 service neutron] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:36.836043 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c04f3e-154f-4700-94bf-8f16d3b003a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:36.837370 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c04f3e-154f-4700-94bf-8f16d3b003a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:36.837660 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c04f3e-154f-4700-94bf-8f16d3b003a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:36.837985 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c04f3e-154f-4700-94bf-8f16d3b003a7 admin admin] PATH_INFO: `/v3/projects/0cc447a829d940d78ebc638f063a3408` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:36.839465 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c04f3e-154f-4700-94bf-8f16d3b003a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cc447a829d940d78ebc638f063a3408)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:36.842925 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c04f3e-154f-4700-94bf-8f16d3b003a7 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:36.878662 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-0416e29b-c8e4-436b-895c-48a609d85a00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '579253bda38141ba973709c0a2412017', 'request_id': 'req-0416e29b-c8e4-436b-895c-48a609d85a00'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:36.886400 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-0416e29b-c8e4-436b-895c-48a609d85a00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '579253bda38141ba973709c0a2412017', 'request_id': 'req-0416e29b-c8e4-436b-895c-48a609d85a00'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:36.892973 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-0416e29b-c8e4-436b-895c-48a609d85a00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '579253bda38141ba973709c0a2412017', 'request_id': 'req-0416e29b-c8e4-436b-895c-48a609d85a00'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:36.971138 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-16c04f3e-154f-4700-94bf-8f16d3b003a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cc447a829d940d78ebc638f063a3408', 'request_id': 'req-16c04f3e-154f-4700-94bf-8f16d3b003a7'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:36.976881 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-16c04f3e-154f-4700-94bf-8f16d3b003a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cc447a829d940d78ebc638f063a3408', 'request_id': 'req-16c04f3e-154f-4700-94bf-8f16d3b003a7'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:36.983076 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-16c04f3e-154f-4700-94bf-8f16d3b003a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cc447a829d940d78ebc638f063a3408', 'request_id': 'req-16c04f3e-154f-4700-94bf-8f16d3b003a7'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:37.019166 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-0416e29b-c8e4-436b-895c-48a609d85a00 admin admin] The token cache is being invalidate because project 579253bda38141ba973709c0a2412017 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:40:37.019166 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-0416e29b-c8e4-436b-895c-48a609d85a00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:37.019166 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-0416e29b-c8e4-436b-895c-48a609d85a00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:37.019166 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1231/2440] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:40:36 2022] DELETE /identity/v3/projects/579253bda38141ba973709c0a2412017 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:40:37.022730 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0416e29b-c8e4-436b-895c-48a609d85a00 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:37.024256 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-0416e29b-c8e4-436b-895c-48a609d85a00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:37.062825 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:37.095731 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0416e29b-c8e4-436b-895c-48a609d85a00 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:37.112214 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-16c04f3e-154f-4700-94bf-8f16d3b003a7 admin admin] The token cache is being invalidate because project 0cc447a829d940d78ebc638f063a3408 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:40:37.112615 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-16c04f3e-154f-4700-94bf-8f16d3b003a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:37.113347 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-16c04f3e-154f-4700-94bf-8f16d3b003a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:37.115954 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1210/2441] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:40:36 2022] DELETE /identity/v3/projects/0cc447a829d940d78ebc638f063a3408 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:40:37.120618 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6687d5-7de4-45d5-81ab-f399817c6d02 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:37.121033 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6687d5-7de4-45d5-81ab-f399817c6d02 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:37.121408 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6687d5-7de4-45d5-81ab-f399817c6d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:37.140565 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-ee6687d5-7de4-45d5-81ab-f399817c6d02 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:37.142049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-ee6687d5-7de4-45d5-81ab-f399817c6d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:37.156157 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589bd0fc-873a-42d9-a7d7-5d560f5f105a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:37.157209 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bd0fc-873a-42d9-a7d7-5d560f5f105a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:37.157562 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bd0fc-873a-42d9-a7d7-5d560f5f105a admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:37.157898 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bd0fc-873a-42d9-a7d7-5d560f5f105a admin admin] PATH_INFO: `/v3/users/c544fc4690a04dad9108f27e6afc1fcc` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:37.159424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589bd0fc-873a-42d9-a7d7-5d560f5f105a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c544fc4690a04dad9108f27e6afc1fcc)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:37.161344 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589bd0fc-873a-42d9-a7d7-5d560f5f105a admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:37.211409 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1211/2442] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:40:37.218698 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e2efc-b3c1-4a6d-be02-1a1205d821f9 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:37.219061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e2efc-b3c1-4a6d-be02-1a1205d821f9 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:37.219402 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e2efc-b3c1-4a6d-be02-1a1205d821f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:37.231915 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:37.238557 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-ee0e2efc-b3c1-4a6d-be02-1a1205d821f9 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:37.240370 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-ee0e2efc-b3c1-4a6d-be02-1a1205d821f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:37.248901 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1212/2443] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:40:37.252957 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e2efc-b3c1-4a6d-be02-1a1205d821f9 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:37.262396 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e2efc-b3c1-4a6d-be02-1a1205d821f9 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:37.267177 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6022fa37-d855-4ade-a76a-9a5bd7ebb8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:37.267943 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022fa37-d855-4ade-a76a-9a5bd7ebb8a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:37.268293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022fa37-d855-4ade-a76a-9a5bd7ebb8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:37.268634 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022fa37-d855-4ade-a76a-9a5bd7ebb8a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:37.269525 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6022fa37-d855-4ade-a76a-9a5bd7ebb8a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:40:37.269867 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6022fa37-d855-4ade-a76a-9a5bd7ebb8a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:37.271547 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6022fa37-d855-4ade-a76a-9a5bd7ebb8a8 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:37.283411 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1213/2444] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:40:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:40:37.287838 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6022fa37-d855-4ade-a76a-9a5bd7ebb8a8 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:37.297367 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6022fa37-d855-4ade-a76a-9a5bd7ebb8a8 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:37.302417 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4490e998-b7e6-470f-a159-32e865dcfe49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:37.303035 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490e998-b7e6-470f-a159-32e865dcfe49 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:37.303306 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490e998-b7e6-470f-a159-32e865dcfe49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:37.303604 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490e998-b7e6-470f-a159-32e865dcfe49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:37.304270 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4490e998-b7e6-470f-a159-32e865dcfe49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:37.306196 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4490e998-b7e6-470f-a159-32e865dcfe49 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:37.332206 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-589bd0fc-873a-42d9-a7d7-5d560f5f105a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c544fc4690a04dad9108f27e6afc1fcc'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:37.332898 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-589bd0fc-873a-42d9-a7d7-5d560f5f105a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c544fc4690a04dad9108f27e6afc1fcc'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:37.340453 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-589bd0fc-873a-42d9-a7d7-5d560f5f105a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c544fc4690a04dad9108f27e6afc1fcc'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:37.341375 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-589bd0fc-873a-42d9-a7d7-5d560f5f105a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c544fc4690a04dad9108f27e6afc1fcc'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:37.354680 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-589bd0fc-873a-42d9-a7d7-5d560f5f105a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c544fc4690a04dad9108f27e6afc1fcc'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:37.403598 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1214/2445] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:40:37 2022] POST /identity/v3/projects => generated 381 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:40:37.405091 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4490e998-b7e6-470f-a159-32e865dcfe49 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:37.406800 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-4490e998-b7e6-470f-a159-32e865dcfe49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:37.460815 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1232/2446] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:40:37 2022] DELETE /identity/v3/users/c544fc4690a04dad9108f27e6afc1fcc => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:40:37.475302 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4490e998-b7e6-470f-a159-32e865dcfe49 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:37.582147 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6deac8-dce9-4f24-b3de-49631e3dd2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:37.583426 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6deac8-dce9-4f24-b3de-49631e3dd2fe admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:37.584170 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6deac8-dce9-4f24-b3de-49631e3dd2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:37.584772 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6deac8-dce9-4f24-b3de-49631e3dd2fe admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:37.586220 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6deac8-dce9-4f24-b3de-49631e3dd2fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:37.589864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6deac8-dce9-4f24-b3de-49631e3dd2fe admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:37.687385 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1215/2447] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:40:37 2022] POST /identity/v3/users => generated 291 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:40:37.689318 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589bd0fc-873a-42d9-a7d7-5d560f5f105a admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:37.701623 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589bd0fc-873a-42d9-a7d7-5d560f5f105a admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:37.707384 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23419891-3578-45ae-8009-fcc12d8c5375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:37.709144 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23419891-3578-45ae-8009-fcc12d8c5375 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:37.709144 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23419891-3578-45ae-8009-fcc12d8c5375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:37.709352 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23419891-3578-45ae-8009-fcc12d8c5375 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:37.710140 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23419891-3578-45ae-8009-fcc12d8c5375 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:37.711860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23419891-3578-45ae-8009-fcc12d8c5375 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:37.724425 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1233/2448] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:40:37.728340 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6deac8-dce9-4f24-b3de-49631e3dd2fe admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:37.738814 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6deac8-dce9-4f24-b3de-49631e3dd2fe admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:37.743809 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d67caa7-2cc3-468e-8636-fd5bf177bcdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:37.744482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67caa7-2cc3-468e-8636-fd5bf177bcdb admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:37.744768 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67caa7-2cc3-468e-8636-fd5bf177bcdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:37.745044 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67caa7-2cc3-468e-8636-fd5bf177bcdb admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:37.745825 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d67caa7-2cc3-468e-8636-fd5bf177bcdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:37.747410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d67caa7-2cc3-468e-8636-fd5bf177bcdb admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:37.758193 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1216/2449] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:40:37.762380 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23419891-3578-45ae-8009-fcc12d8c5375 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:37.773413 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23419891-3578-45ae-8009-fcc12d8c5375 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:37.778493 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b896d-95fd-43f4-85b2-f7dc2384d349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:37.779384 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b896d-95fd-43f4-85b2-f7dc2384d349 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:37.779710 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b896d-95fd-43f4-85b2-f7dc2384d349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:37.779989 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b896d-95fd-43f4-85b2-f7dc2384d349 admin admin] PATH_INFO: `/v3/projects/2f68e6b7c9dc49dc86e3296517db6b50/users/282ba7270e784402af6f0763fb6263e8/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:37.797202 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215b896d-95fd-43f4-85b2-f7dc2384d349 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f68e6b7c9dc49dc86e3296517db6b50, user_id=282ba7270e784402af6f0763fb6263e8, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:37.799970 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215b896d-95fd-43f4-85b2-f7dc2384d349 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:37.871607 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1234/2450] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:40:37 2022] PUT /identity/v3/projects/2f68e6b7c9dc49dc86e3296517db6b50/users/282ba7270e784402af6f0763fb6263e8/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:40:37.883202 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b806b-c683-4c06-92cd-5de9955e5e16 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:37.883748 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b806b-c683-4c06-92cd-5de9955e5e16 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:37.883869 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b806b-c683-4c06-92cd-5de9955e5e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:37.912993 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-856b806b-c683-4c06-92cd-5de9955e5e16 None None] MFA Rules not processed for user `282ba7270e784402af6f0763fb6263e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:37.914365 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-856b806b-c683-4c06-92cd-5de9955e5e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:37.967498 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:37.992590 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1217/2451] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:40:37.997186 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b896d-95fd-43f4-85b2-f7dc2384d349 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:38.018634 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:38.018860 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b806b-c683-4c06-92cd-5de9955e5e16 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:38.021144 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-856b806b-c683-4c06-92cd-5de9955e5e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:38.070592 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b896d-95fd-43f4-85b2-f7dc2384d349 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:38.075416 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ff35e-a096-41cd-ae5e-9cef68910b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:38.076158 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ff35e-a096-41cd-ae5e-9cef68910b6a admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:38.076553 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ff35e-a096-41cd-ae5e-9cef68910b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:38.076822 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ff35e-a096-41cd-ae5e-9cef68910b6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:38.077453 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269ff35e-a096-41cd-ae5e-9cef68910b6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:38.079376 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269ff35e-a096-41cd-ae5e-9cef68910b6a admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:38.095348 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b806b-c683-4c06-92cd-5de9955e5e16 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:38.100727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e45b6-c794-4a5f-886a-438ac93e96b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:38.101579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e45b6-c794-4a5f-886a-438ac93e96b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:38.101954 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e45b6-c794-4a5f-886a-438ac93e96b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:38.102186 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e45b6-c794-4a5f-886a-438ac93e96b4 admin admin] PATH_INFO: `/v3/projects/2dfe442d438f49549480ff42ca075de6` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:38.103368 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1e45b6-c794-4a5f-886a-438ac93e96b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dfe442d438f49549480ff42ca075de6)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:38.105412 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1e45b6-c794-4a5f-886a-438ac93e96b4 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:38.211991 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1235/2452] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:40:37 2022] POST /identity/v3/projects => generated 381 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:40:38.216082 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ff35e-a096-41cd-ae5e-9cef68910b6a admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:38.254446 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-1b1e45b6-c794-4a5f-886a-438ac93e96b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dfe442d438f49549480ff42ca075de6', 'request_id': 'req-1b1e45b6-c794-4a5f-886a-438ac93e96b4'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:38.261524 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-1b1e45b6-c794-4a5f-886a-438ac93e96b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dfe442d438f49549480ff42ca075de6', 'request_id': 'req-1b1e45b6-c794-4a5f-886a-438ac93e96b4'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:38.268191 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-1b1e45b6-c794-4a5f-886a-438ac93e96b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dfe442d438f49549480ff42ca075de6', 'request_id': 'req-1b1e45b6-c794-4a5f-886a-438ac93e96b4'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:38.291990 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ff35e-a096-41cd-ae5e-9cef68910b6a admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:38.350717 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d9d0fc-ac92-49d0-b0f4-b1be36181b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:38.351395 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9d0fc-ac92-49d0-b0f4-b1be36181b9e admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:38.351711 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9d0fc-ac92-49d0-b0f4-b1be36181b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:38.351988 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9d0fc-ac92-49d0-b0f4-b1be36181b9e admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:38.352104 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-1b1e45b6-c794-4a5f-886a-438ac93e96b4 admin admin] The token cache is being invalidate because project 2dfe442d438f49549480ff42ca075de6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:40:38.352359 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-1b1e45b6-c794-4a5f-886a-438ac93e96b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:38.352827 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d9d0fc-ac92-49d0-b0f4-b1be36181b9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:38.353129 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-1b1e45b6-c794-4a5f-886a-438ac93e96b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:38.354706 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1218/2453] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:40:38 2022] DELETE /identity/v3/projects/2dfe442d438f49549480ff42ca075de6 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:40:38.354874 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d9d0fc-ac92-49d0-b0f4-b1be36181b9e admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:38.359536 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6beaab-c94c-49df-95a0-912cfc5dafb7 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:38.359885 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6beaab-c94c-49df-95a0-912cfc5dafb7 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:38.360344 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6beaab-c94c-49df-95a0-912cfc5dafb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:38.380300 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-1b6beaab-c94c-49df-95a0-912cfc5dafb7 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:38.381594 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-1b6beaab-c94c-49df-95a0-912cfc5dafb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:38.389693 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1219/2454] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:40:38.397827 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3284ef-9fc6-4603-b8a4-2caf36086205 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:38.398109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3284ef-9fc6-4603-b8a4-2caf36086205 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:38.398381 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3284ef-9fc6-4603-b8a4-2caf36086205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:38.415546 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-ad3284ef-9fc6-4603-b8a4-2caf36086205 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:38.416727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-ad3284ef-9fc6-4603-b8a4-2caf36086205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:38.424434 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1220/2455] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:40:38.428407 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3284ef-9fc6-4603-b8a4-2caf36086205 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:38.437460 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3284ef-9fc6-4603-b8a4-2caf36086205 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:38.441962 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b461af26-e4c4-492f-8562-50096c79f5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:38.443174 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461af26-e4c4-492f-8562-50096c79f5f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:38.443453 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461af26-e4c4-492f-8562-50096c79f5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:38.443763 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461af26-e4c4-492f-8562-50096c79f5f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:38.448278 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b461af26-e4c4-492f-8562-50096c79f5f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:40:38.448278 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b461af26-e4c4-492f-8562-50096c79f5f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:38.448278 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b461af26-e4c4-492f-8562-50096c79f5f7 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:38.458646 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1221/2456] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:40:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:40:38.462710 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b461af26-e4c4-492f-8562-50096c79f5f7 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:38.472223 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b461af26-e4c4-492f-8562-50096c79f5f7 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:38.473712 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1236/2457] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:40:38 2022] POST /identity/v3/users => generated 297 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:40:38.477483 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07ac27-e4bb-4f21-add1-f04efbc1ac25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:38.478127 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07ac27-e4bb-4f21-add1-f04efbc1ac25 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:38.478411 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07ac27-e4bb-4f21-add1-f04efbc1ac25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:38.478719 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07ac27-e4bb-4f21-add1-f04efbc1ac25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:38.479407 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d07ac27-e4bb-4f21-add1-f04efbc1ac25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:38.480712 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d9d0fc-ac92-49d0-b0f4-b1be36181b9e admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:38.481331 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d07ac27-e4bb-4f21-add1-f04efbc1ac25 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:38.482393 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-99d9d0fc-ac92-49d0-b0f4-b1be36181b9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:38.495392 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d9d0fc-ac92-49d0-b0f4-b1be36181b9e admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:38.500183 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a400f6a-997f-44ab-89e6-088cdffe4f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:38.500852 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a400f6a-997f-44ab-89e6-088cdffe4f23 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:38.501134 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a400f6a-997f-44ab-89e6-088cdffe4f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:38.501414 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a400f6a-997f-44ab-89e6-088cdffe4f23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:38.502147 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a400f6a-997f-44ab-89e6-088cdffe4f23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:38.503824 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a400f6a-997f-44ab-89e6-088cdffe4f23 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:38.514371 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1237/2458] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:40:38.518603 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a400f6a-997f-44ab-89e6-088cdffe4f23 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:38.528031 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a400f6a-997f-44ab-89e6-088cdffe4f23 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:38.533991 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f84cb-f5db-4452-b427-397f1254ea63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:38.534869 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f84cb-f5db-4452-b427-397f1254ea63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:38.535199 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f84cb-f5db-4452-b427-397f1254ea63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:38.535553 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f84cb-f5db-4452-b427-397f1254ea63 admin admin] PATH_INFO: `/v3/projects/6e273d007934466caa5503738d5c59b1/users/d6ee81fbd7d54838b7845e16e9ab985a/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:38.553343 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f84cb-f5db-4452-b427-397f1254ea63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e273d007934466caa5503738d5c59b1, user_id=d6ee81fbd7d54838b7845e16e9ab985a, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:38.558381 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1222/2459] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:40:38 2022] POST /identity/v3/projects => generated 363 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:40:38.558533 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f84cb-f5db-4452-b427-397f1254ea63 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:38.559490 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07ac27-e4bb-4f21-add1-f04efbc1ac25 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:38.589143 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:38.625738 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07ac27-e4bb-4f21-add1-f04efbc1ac25 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:38.631252 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141afb67-874f-47b3-bff7-1d8f814e3353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:38.631995 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141afb67-874f-47b3-bff7-1d8f814e3353 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:38.632285 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141afb67-874f-47b3-bff7-1d8f814e3353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:38.632565 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141afb67-874f-47b3-bff7-1d8f814e3353 admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:38.633335 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141afb67-874f-47b3-bff7-1d8f814e3353 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:38.635292 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141afb67-874f-47b3-bff7-1d8f814e3353 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:38.644365 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1238/2460] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:40:38 2022] PUT /identity/v3/projects/6e273d007934466caa5503738d5c59b1/users/d6ee81fbd7d54838b7845e16e9ab985a/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:40:38.648630 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f84cb-f5db-4452-b427-397f1254ea63 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:38.657203 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:38.710688 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f84cb-f5db-4452-b427-397f1254ea63 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:38.715307 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee82b1-a4b5-4e6f-81f8-6ae99cccdb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:38.715999 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee82b1-a4b5-4e6f-81f8-6ae99cccdb9f admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:38.716283 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee82b1-a4b5-4e6f-81f8-6ae99cccdb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:38.716560 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee82b1-a4b5-4e6f-81f8-6ae99cccdb9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:38.717279 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bee82b1-a4b5-4e6f-81f8-6ae99cccdb9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:38.718869 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bee82b1-a4b5-4e6f-81f8-6ae99cccdb9f admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:38.728500 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1239/2461] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:40:38.732491 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee82b1-a4b5-4e6f-81f8-6ae99cccdb9f admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:38.745023 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1223/2462] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:40:38 2022] POST /identity/v3/users => generated 282 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:40:38.745247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee82b1-a4b5-4e6f-81f8-6ae99cccdb9f admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:38.746015 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141afb67-874f-47b3-bff7-1d8f814e3353 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:38.748011 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c8e93-2b95-440f-8330-ea1cf34a7602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:38.748874 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c8e93-2b95-440f-8330-ea1cf34a7602 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:38.749152 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c8e93-2b95-440f-8330-ea1cf34a7602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:38.749426 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c8e93-2b95-440f-8330-ea1cf34a7602 admin admin] PATH_INFO: `/v3/projects/6e273d007934466caa5503738d5c59b1/users/d6ee81fbd7d54838b7845e16e9ab985a/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:38.751385 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c8e93-2b95-440f-8330-ea1cf34a7602 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e273d007934466caa5503738d5c59b1, user_id=d6ee81fbd7d54838b7845e16e9ab985a, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:38.754127 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c8e93-2b95-440f-8330-ea1cf34a7602 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:38.758312 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141afb67-874f-47b3-bff7-1d8f814e3353 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:38.761126 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e860e92-9626-42e5-aaef-372811516b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:38.761817 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e860e92-9626-42e5-aaef-372811516b57 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:38.762103 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e860e92-9626-42e5-aaef-372811516b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:38.762383 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e860e92-9626-42e5-aaef-372811516b57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:38.763207 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e860e92-9626-42e5-aaef-372811516b57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:38.764916 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e860e92-9626-42e5-aaef-372811516b57 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:38.765840 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1240/2463] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:40:38 2022] PUT /identity/v3/projects/6e273d007934466caa5503738d5c59b1/users/d6ee81fbd7d54838b7845e16e9ab985a/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:40:38.777988 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1224/2464] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:40:38.778158 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b9aa3-bb28-4723-b0c1-531a82c506f0 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:38.778158 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b9aa3-bb28-4723-b0c1-531a82c506f0 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:38.778158 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b9aa3-bb28-4723-b0c1-531a82c506f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:38.778911 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e860e92-9626-42e5-aaef-372811516b57 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:38.788878 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e860e92-9626-42e5-aaef-372811516b57 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:38.803373 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-564b9aa3-bb28-4723-b0c1-531a82c506f0 None None] MFA Rules not processed for user `d6ee81fbd7d54838b7845e16e9ab985a`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:38.810327 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-564b9aa3-bb28-4723-b0c1-531a82c506f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:38.846118 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0d105c-6267-449a-a678-3da6bc6bf3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:38.846780 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d105c-6267-449a-a678-3da6bc6bf3f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:38.847062 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d105c-6267-449a-a678-3da6bc6bf3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:38.847336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d105c-6267-449a-a678-3da6bc6bf3f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:38.848087 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0d105c-6267-449a-a678-3da6bc6bf3f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:38.850169 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0d105c-6267-449a-a678-3da6bc6bf3f9 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:38.863529 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1225/2465] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:40:38.865115 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0d105c-6267-449a-a678-3da6bc6bf3f9 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:38.874756 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0d105c-6267-449a-a678-3da6bc6bf3f9 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:38.878830 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1241/2466] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:38 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:40:38.880425 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11f58d1-a6a7-4b9b-b775-c26b029c56f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:38.881306 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f58d1-a6a7-4b9b-b775-c26b029c56f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:38.881588 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f58d1-a6a7-4b9b-b775-c26b029c56f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:38.881972 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f58d1-a6a7-4b9b-b775-c26b029c56f1 admin admin] PATH_INFO: `/v3/projects/9dc341221aa64bdfbdba2ae03dc01ee8/users/617d753fb0c84bcf905d57d385b8e52f/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:38.889095 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b9aa3-bb28-4723-b0c1-531a82c506f0 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:38.890503 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-564b9aa3-bb28-4723-b0c1-531a82c506f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:38.898026 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11f58d1-a6a7-4b9b-b775-c26b029c56f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dc341221aa64bdfbdba2ae03dc01ee8, user_id=617d753fb0c84bcf905d57d385b8e52f, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:38.900779 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11f58d1-a6a7-4b9b-b775-c26b029c56f1 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:38.946366 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b9aa3-bb28-4723-b0c1-531a82c506f0 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:38.949884 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad5ae3-417a-40c6-ba89-2e7fb6d1cc49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:38.950522 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad5ae3-417a-40c6-ba89-2e7fb6d1cc49 service cinder] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:38.950801 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad5ae3-417a-40c6-ba89-2e7fb6d1cc49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:38.951073 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad5ae3-417a-40c6-ba89-2e7fb6d1cc49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:38.960217 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad5ae3-417a-40c6-ba89-2e7fb6d1cc49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:38.961688 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad5ae3-417a-40c6-ba89-2e7fb6d1cc49 service cinder] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:38.971419 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1242/2467] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:40:38 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 15:40:38.997506 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1226/2468] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:40:38 2022] PUT /identity/v3/projects/9dc341221aa64bdfbdba2ae03dc01ee8/users/617d753fb0c84bcf905d57d385b8e52f/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:40:39.007731 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86fc579-72a1-444d-802c-fbd6958a71c4 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:39.008015 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86fc579-72a1-444d-802c-fbd6958a71c4 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:39.008311 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86fc579-72a1-444d-802c-fbd6958a71c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:39.033766 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-b86fc579-72a1-444d-802c-fbd6958a71c4 None None] MFA Rules not processed for user `617d753fb0c84bcf905d57d385b8e52f`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:39.035050 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-b86fc579-72a1-444d-802c-fbd6958a71c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:39.040492 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:39.101849 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1243/2469] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:39 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:40:39.106476 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11f58d1-a6a7-4b9b-b775-c26b029c56f1 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:39.169363 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11f58d1-a6a7-4b9b-b775-c26b029c56f1 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:39.174377 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ab648-78bc-4048-a956-b9b4c759c180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:39.175039 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ab648-78bc-4048-a956-b9b4c759c180 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:39.175337 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ab648-78bc-4048-a956-b9b4c759c180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:39.175760 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ab648-78bc-4048-a956-b9b4c759c180 admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:39.176455 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ab648-78bc-4048-a956-b9b4c759c180 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:39.183680 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ab648-78bc-4048-a956-b9b4c759c180 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:39.190826 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86fc579-72a1-444d-802c-fbd6958a71c4 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:39.191693 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-b86fc579-72a1-444d-802c-fbd6958a71c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:39.258872 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86fc579-72a1-444d-802c-fbd6958a71c4 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:39.265879 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a89cb2e-f2f9-45f2-ba47-2c8d58015e03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:39.266668 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a89cb2e-f2f9-45f2-ba47-2c8d58015e03 service cinder] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:39.267098 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a89cb2e-f2f9-45f2-ba47-2c8d58015e03 service cinder] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:39.267373 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a89cb2e-f2f9-45f2-ba47-2c8d58015e03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:39.269344 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-7a89cb2e-f2f9-45f2-ba47-2c8d58015e03 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:39.301861 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:39.312701 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1227/2470] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:40:39 2022] POST /identity/v3/projects => generated 363 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:40:39.313671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ab648-78bc-4048-a956-b9b4c759c180 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:39.332627 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a89cb2e-f2f9-45f2-ba47-2c8d58015e03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:39.334586 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a89cb2e-f2f9-45f2-ba47-2c8d58015e03 service cinder] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:39.343556 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1244/2471] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:40:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 15:40:39.380446 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ab648-78bc-4048-a956-b9b4c759c180 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:39.385921 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6863cd-e59c-4f22-9e30-dc71c1e539c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:39.386207 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6863cd-e59c-4f22-9e30-dc71c1e539c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:39.386491 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6863cd-e59c-4f22-9e30-dc71c1e539c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:39.386764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6863cd-e59c-4f22-9e30-dc71c1e539c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:39.387448 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6863cd-e59c-4f22-9e30-dc71c1e539c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:39.389400 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6863cd-e59c-4f22-9e30-dc71c1e539c1 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:39.486392 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1228/2472] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:40:39 2022] POST /identity/v3/users => generated 288 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:40:39.490314 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a89cb2e-f2f9-45f2-ba47-2c8d58015e03 service cinder] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:39.501004 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a89cb2e-f2f9-45f2-ba47-2c8d58015e03 service cinder] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:39.505575 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1c762-53e1-4f04-85d1-21ce645009f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:39.506232 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1c762-53e1-4f04-85d1-21ce645009f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:39.506517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1c762-53e1-4f04-85d1-21ce645009f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:39.506795 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1c762-53e1-4f04-85d1-21ce645009f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:39.507517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d1c762-53e1-4f04-85d1-21ce645009f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:39.509130 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d1c762-53e1-4f04-85d1-21ce645009f6 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:39.518918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1245/2473] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:40:39.523487 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6863cd-e59c-4f22-9e30-dc71c1e539c1 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:39.534669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6863cd-e59c-4f22-9e30-dc71c1e539c1 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:39.540423 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fc5969-d3bb-4dad-b1f4-144e46fd02fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:39.541350 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc5969-d3bb-4dad-b1f4-144e46fd02fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:39.541627 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc5969-d3bb-4dad-b1f4-144e46fd02fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:39.541895 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc5969-d3bb-4dad-b1f4-144e46fd02fb admin admin] PATH_INFO: `/v3/projects/c882e2fdad11420eaabfd44815efe0cc/users/151884a2fa0844269ce91087c839d36a/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:39.558308 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fc5969-d3bb-4dad-b1f4-144e46fd02fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c882e2fdad11420eaabfd44815efe0cc, user_id=151884a2fa0844269ce91087c839d36a, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:39.561005 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fc5969-d3bb-4dad-b1f4-144e46fd02fb admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:39.627783 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1229/2474] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:40:39 2022] PUT /identity/v3/projects/c882e2fdad11420eaabfd44815efe0cc/users/151884a2fa0844269ce91087c839d36a/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:40:39.632453 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1c762-53e1-4f04-85d1-21ce645009f6 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:39.642984 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1c762-53e1-4f04-85d1-21ce645009f6 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:39.700822 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:39.713819 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa35b8ec-39cd-4a0c-bb70-d62a74674c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:39.714418 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa35b8ec-39cd-4a0c-bb70-d62a74674c23 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:39.714697 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa35b8ec-39cd-4a0c-bb70-d62a74674c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:39.714975 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa35b8ec-39cd-4a0c-bb70-d62a74674c23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:39.715732 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa35b8ec-39cd-4a0c-bb70-d62a74674c23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:39.717417 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa35b8ec-39cd-4a0c-bb70-d62a74674c23 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:39.727558 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1246/2475] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:40:39.737271 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fc5969-d3bb-4dad-b1f4-144e46fd02fb admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:39.748190 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fc5969-d3bb-4dad-b1f4-144e46fd02fb admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:39.755532 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb113c2-bdd5-4748-8bc8-06a149f02209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:39.756443 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb113c2-bdd5-4748-8bc8-06a149f02209 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:39.756867 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb113c2-bdd5-4748-8bc8-06a149f02209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:39.757115 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb113c2-bdd5-4748-8bc8-06a149f02209 admin admin] PATH_INFO: `/v3/projects/c882e2fdad11420eaabfd44815efe0cc/users/151884a2fa0844269ce91087c839d36a/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:39.759230 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb113c2-bdd5-4748-8bc8-06a149f02209 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c882e2fdad11420eaabfd44815efe0cc, user_id=151884a2fa0844269ce91087c839d36a, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:39.762102 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb113c2-bdd5-4748-8bc8-06a149f02209 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:39.774564 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1230/2476] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:40:39 2022] PUT /identity/v3/projects/c882e2fdad11420eaabfd44815efe0cc/users/151884a2fa0844269ce91087c839d36a/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:40:39.787718 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d863e9f-36a6-4102-81b1-e1d0cd6d384d None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:39.788015 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d863e9f-36a6-4102-81b1-e1d0cd6d384d None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:39.788322 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d863e9f-36a6-4102-81b1-e1d0cd6d384d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:39.818007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-4d863e9f-36a6-4102-81b1-e1d0cd6d384d None None] MFA Rules not processed for user `151884a2fa0844269ce91087c839d36a`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:39.819275 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-4d863e9f-36a6-4102-81b1-e1d0cd6d384d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:39.888330 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1247/2477] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:39 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:40:39.900376 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb113c2-bdd5-4748-8bc8-06a149f02209 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:39.958560 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb113c2-bdd5-4748-8bc8-06a149f02209 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:39.962666 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ee49f-b322-42c5-af24-d36451df7631 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:39.963531 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ee49f-b322-42c5-af24-d36451df7631 service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:39.963924 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ee49f-b322-42c5-af24-d36451df7631 service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:39.964213 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ee49f-b322-42c5-af24-d36451df7631 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:39.974035 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ee49f-b322-42c5-af24-d36451df7631 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:39.975457 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ee49f-b322-42c5-af24-d36451df7631 service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:39.984981 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1231/2478] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:40:39 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:40:40.589624 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d863e9f-36a6-4102-81b1-e1d0cd6d384d None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:40.591030 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-4d863e9f-36a6-4102-81b1-e1d0cd6d384d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:40.650944 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d863e9f-36a6-4102-81b1-e1d0cd6d384d None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:40.654536 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb70444-a43f-446a-a121-459070274bf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:40.655181 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb70444-a43f-446a-a121-459070274bf1 service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:40.655462 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb70444-a43f-446a-a121-459070274bf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:40.655768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb70444-a43f-446a-a121-459070274bf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:40.659978 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-feb70444-a43f-446a-a121-459070274bf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:40.723387 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb70444-a43f-446a-a121-459070274bf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:40.725511 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb70444-a43f-446a-a121-459070274bf1 service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:40.735411 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1248/2479] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:40:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:40:40.861850 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ee49f-b322-42c5-af24-d36451df7631 service cinder] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:40.875020 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ee49f-b322-42c5-af24-d36451df7631 service cinder] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:40.880133 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab27e6-c0c1-462f-a130-7d4b47a276d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:40.881172 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab27e6-c0c1-462f-a130-7d4b47a276d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:40.881528 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab27e6-c0c1-462f-a130-7d4b47a276d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:40.881880 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab27e6-c0c1-462f-a130-7d4b47a276d0 admin admin] PATH_INFO: `/v3/users/617d753fb0c84bcf905d57d385b8e52f` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:40.883451 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab27e6-c0c1-462f-a130-7d4b47a276d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=617d753fb0c84bcf905d57d385b8e52f)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:40.885539 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab27e6-c0c1-462f-a130-7d4b47a276d0 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:41.040870 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-4dab27e6-c0c1-462f-a130-7d4b47a276d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '617d753fb0c84bcf905d57d385b8e52f'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:41.042094 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:41.048580 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-4dab27e6-c0c1-462f-a130-7d4b47a276d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '617d753fb0c84bcf905d57d385b8e52f'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:41.062767 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-4dab27e6-c0c1-462f-a130-7d4b47a276d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '617d753fb0c84bcf905d57d385b8e52f'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:41.063578 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-4dab27e6-c0c1-462f-a130-7d4b47a276d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '617d753fb0c84bcf905d57d385b8e52f'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:41.169703 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-4dab27e6-c0c1-462f-a130-7d4b47a276d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '617d753fb0c84bcf905d57d385b8e52f'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:41.176911 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1232/2480] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:40:40 2022] DELETE /identity/v3/users/617d753fb0c84bcf905d57d385b8e52f => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:40:41.181222 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbabfb-bb92-44cb-b1e2-687f65fe62a3 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:41.181521 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbabfb-bb92-44cb-b1e2-687f65fe62a3 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:41.181788 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbabfb-bb92-44cb-b1e2-687f65fe62a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:41.202498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-05bbabfb-bb92-44cb-b1e2-687f65fe62a3 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:41.203751 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-05bbabfb-bb92-44cb-b1e2-687f65fe62a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:41.221326 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:41.264946 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1249/2481] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:40:41.275679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab27e6-c0c1-462f-a130-7d4b47a276d0 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:41.277407 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-4dab27e6-c0c1-462f-a130-7d4b47a276d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:41.336033 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab27e6-c0c1-462f-a130-7d4b47a276d0 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:41.340111 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1482417-b00e-4ebe-8fff-e2069f8a36a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:41.340760 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1482417-b00e-4ebe-8fff-e2069f8a36a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:41.341044 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1482417-b00e-4ebe-8fff-e2069f8a36a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:41.341327 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1482417-b00e-4ebe-8fff-e2069f8a36a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:41.350503 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1482417-b00e-4ebe-8fff-e2069f8a36a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:41.351967 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1482417-b00e-4ebe-8fff-e2069f8a36a8 service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:41.362446 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1233/2482] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:40:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:40:41.938383 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bbabfb-bb92-44cb-b1e2-687f65fe62a3 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:41.939926 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-05bbabfb-bb92-44cb-b1e2-687f65fe62a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:41.956569 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bbabfb-bb92-44cb-b1e2-687f65fe62a3 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:41.967847 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba46c869-9bd0-4d86-b9d0-ee810ea823b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:41.970379 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46c869-9bd0-4d86-b9d0-ee810ea823b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:41.970379 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46c869-9bd0-4d86-b9d0-ee810ea823b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:41.970379 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46c869-9bd0-4d86-b9d0-ee810ea823b1 admin admin] PATH_INFO: `/v3/projects/9dc341221aa64bdfbdba2ae03dc01ee8` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:41.970640 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba46c869-9bd0-4d86-b9d0-ee810ea823b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dc341221aa64bdfbdba2ae03dc01ee8)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:41.972434 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba46c869-9bd0-4d86-b9d0-ee810ea823b1 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:42.068830 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-ba46c869-9bd0-4d86-b9d0-ee810ea823b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dc341221aa64bdfbdba2ae03dc01ee8', 'request_id': 'req-ba46c869-9bd0-4d86-b9d0-ee810ea823b1'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:42.074404 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-ba46c869-9bd0-4d86-b9d0-ee810ea823b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dc341221aa64bdfbdba2ae03dc01ee8', 'request_id': 'req-ba46c869-9bd0-4d86-b9d0-ee810ea823b1'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:42.080717 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-ba46c869-9bd0-4d86-b9d0-ee810ea823b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dc341221aa64bdfbdba2ae03dc01ee8', 'request_id': 'req-ba46c869-9bd0-4d86-b9d0-ee810ea823b1'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:42.151758 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-ba46c869-9bd0-4d86-b9d0-ee810ea823b1 admin admin] The token cache is being invalidate because project 9dc341221aa64bdfbdba2ae03dc01ee8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:40:42.152050 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-ba46c869-9bd0-4d86-b9d0-ee810ea823b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:42.152816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-ba46c869-9bd0-4d86-b9d0-ee810ea823b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:42.154448 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1250/2483] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:40:41 2022] DELETE /identity/v3/projects/9dc341221aa64bdfbdba2ae03dc01ee8 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:40:42.158448 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1482417-b00e-4ebe-8fff-e2069f8a36a8 service neutron] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:42.168915 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1482417-b00e-4ebe-8fff-e2069f8a36a8 service neutron] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:42.235455 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b9a77-9db1-4e01-b94b-cb9702489fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:42.237635 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b9a77-9db1-4e01-b94b-cb9702489fb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:42.237992 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b9a77-9db1-4e01-b94b-cb9702489fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:42.238491 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b9a77-9db1-4e01-b94b-cb9702489fb7 admin admin] PATH_INFO: `/v3/users/151884a2fa0844269ce91087c839d36a` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:42.240086 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765b9a77-9db1-4e01-b94b-cb9702489fb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=151884a2fa0844269ce91087c839d36a)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:42.242499 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765b9a77-9db1-4e01-b94b-cb9702489fb7 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:42.398451 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-765b9a77-9db1-4e01-b94b-cb9702489fb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '151884a2fa0844269ce91087c839d36a'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:42.406685 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-765b9a77-9db1-4e01-b94b-cb9702489fb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '151884a2fa0844269ce91087c839d36a'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:42.420190 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:42.422713 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-765b9a77-9db1-4e01-b94b-cb9702489fb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '151884a2fa0844269ce91087c839d36a'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:42.423551 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-765b9a77-9db1-4e01-b94b-cb9702489fb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '151884a2fa0844269ce91087c839d36a'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:42.500079 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-765b9a77-9db1-4e01-b94b-cb9702489fb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '151884a2fa0844269ce91087c839d36a'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:42.507247 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1234/2484] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:40:42 2022] DELETE /identity/v3/users/151884a2fa0844269ce91087c839d36a => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:40:43.151743 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba46c869-9bd0-4d86-b9d0-ee810ea823b1 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:43.153268 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-ba46c869-9bd0-4d86-b9d0-ee810ea823b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:43.169227 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba46c869-9bd0-4d86-b9d0-ee810ea823b1 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:43.236364 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cf9570-4159-4403-be3f-afb749d44340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:43.237240 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf9570-4159-4403-be3f-afb749d44340 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:43.237519 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf9570-4159-4403-be3f-afb749d44340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:43.237786 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf9570-4159-4403-be3f-afb749d44340 admin admin] PATH_INFO: `/v3/projects/c882e2fdad11420eaabfd44815efe0cc` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:43.239507 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cf9570-4159-4403-be3f-afb749d44340 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c882e2fdad11420eaabfd44815efe0cc)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:43.241492 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cf9570-4159-4403-be3f-afb749d44340 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:43.354683 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:43.356018 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-f7cf9570-4159-4403-be3f-afb749d44340 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c882e2fdad11420eaabfd44815efe0cc', 'request_id': 'req-f7cf9570-4159-4403-be3f-afb749d44340'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:43.362392 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-f7cf9570-4159-4403-be3f-afb749d44340 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c882e2fdad11420eaabfd44815efe0cc', 'request_id': 'req-f7cf9570-4159-4403-be3f-afb749d44340'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:43.371393 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-f7cf9570-4159-4403-be3f-afb749d44340 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c882e2fdad11420eaabfd44815efe0cc', 'request_id': 'req-f7cf9570-4159-4403-be3f-afb749d44340'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:43.427652 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-f7cf9570-4159-4403-be3f-afb749d44340 admin admin] The token cache is being invalidate because project c882e2fdad11420eaabfd44815efe0cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:40:43.428259 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-f7cf9570-4159-4403-be3f-afb749d44340 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:43.429778 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-f7cf9570-4159-4403-be3f-afb749d44340 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:43.431667 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1251/2485] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:40:43 2022] DELETE /identity/v3/projects/c882e2fdad11420eaabfd44815efe0cc => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:40:43.436491 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7f43a-b57b-408c-8fa2-e83e18d9bce9 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:43.436936 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7f43a-b57b-408c-8fa2-e83e18d9bce9 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:43.437043 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7f43a-b57b-408c-8fa2-e83e18d9bce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:43.458187 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-a6e7f43a-b57b-408c-8fa2-e83e18d9bce9 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:43.459582 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-a6e7f43a-b57b-408c-8fa2-e83e18d9bce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:43.589241 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1235/2486] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:40:43.597142 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74027d4e-3ecd-40d4-9b79-a46b7f4924a8 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:43.597431 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74027d4e-3ecd-40d4-9b79-a46b7f4924a8 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:43.597705 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74027d4e-3ecd-40d4-9b79-a46b7f4924a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:43.617121 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-74027d4e-3ecd-40d4-9b79-a46b7f4924a8 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:43.618367 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-74027d4e-3ecd-40d4-9b79-a46b7f4924a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:43.626222 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1252/2487] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:40:43.630573 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e7f43a-b57b-408c-8fa2-e83e18d9bce9 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:43.640323 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e7f43a-b57b-408c-8fa2-e83e18d9bce9 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:43.645004 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6df03-f048-4338-a456-aa5e1ad6af52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:43.645638 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6df03-f048-4338-a456-aa5e1ad6af52 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:43.645926 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6df03-f048-4338-a456-aa5e1ad6af52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:43.646212 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6df03-f048-4338-a456-aa5e1ad6af52 admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:43.646939 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f6df03-f048-4338-a456-aa5e1ad6af52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:40:43.647250 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f6df03-f048-4338-a456-aa5e1ad6af52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:43.648920 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f6df03-f048-4338-a456-aa5e1ad6af52 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:43.666372 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1236/2488] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:40:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:40:43.670912 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74027d4e-3ecd-40d4-9b79-a46b7f4924a8 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:43.682117 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74027d4e-3ecd-40d4-9b79-a46b7f4924a8 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:43.688140 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b3fe3-6cd3-41d2-a352-8e011aa517df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:43.688806 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b3fe3-6cd3-41d2-a352-8e011aa517df admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:43.689082 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b3fe3-6cd3-41d2-a352-8e011aa517df admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:43.689380 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b3fe3-6cd3-41d2-a352-8e011aa517df admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:43.690036 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002b3fe3-6cd3-41d2-a352-8e011aa517df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:43.691998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002b3fe3-6cd3-41d2-a352-8e011aa517df admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:43.807918 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1253/2489] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:40:43 2022] POST /identity/v3/projects => generated 385 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:40:43.812177 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6df03-f048-4338-a456-aa5e1ad6af52 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:43.822853 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6df03-f048-4338-a456-aa5e1ad6af52 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:43.884656 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8377f-113f-4dc2-91cf-bdccff544a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:43.885329 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8377f-113f-4dc2-91cf-bdccff544a12 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:43.885615 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8377f-113f-4dc2-91cf-bdccff544a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:43.885912 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8377f-113f-4dc2-91cf-bdccff544a12 admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:43.886594 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b8377f-113f-4dc2-91cf-bdccff544a12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:43.888551 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b8377f-113f-4dc2-91cf-bdccff544a12 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:43.970095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1237/2490] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:40:43 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:40:43.975203 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b3fe3-6cd3-41d2-a352-8e011aa517df admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:43.987967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b3fe3-6cd3-41d2-a352-8e011aa517df admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:43.994150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0032a87-c405-4df0-98c4-cfe8bd73319f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:43.995042 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0032a87-c405-4df0-98c4-cfe8bd73319f admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:43.995322 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0032a87-c405-4df0-98c4-cfe8bd73319f admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:43.995757 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0032a87-c405-4df0-98c4-cfe8bd73319f admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:43.996614 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0032a87-c405-4df0-98c4-cfe8bd73319f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:43.998614 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0032a87-c405-4df0-98c4-cfe8bd73319f admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:44.013135 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1254/2491] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:40:44.014461 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8377f-113f-4dc2-91cf-bdccff544a12 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:44.025210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8377f-113f-4dc2-91cf-bdccff544a12 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:44.029836 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3457709-7fee-4047-8dac-953c1d1f97d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:44.030495 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3457709-7fee-4047-8dac-953c1d1f97d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:44.030779 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3457709-7fee-4047-8dac-953c1d1f97d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:44.031112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3457709-7fee-4047-8dac-953c1d1f97d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:44.031893 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3457709-7fee-4047-8dac-953c1d1f97d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:44.033510 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3457709-7fee-4047-8dac-953c1d1f97d0 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:44.048085 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1238/2492] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:40:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:40:44.049302 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0032a87-c405-4df0-98c4-cfe8bd73319f admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:44.060388 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0032a87-c405-4df0-98c4-cfe8bd73319f admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:44.065874 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57772d8-60ff-4486-ab5f-25d0699e3736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:44.066611 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57772d8-60ff-4486-ab5f-25d0699e3736 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:44.067287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57772d8-60ff-4486-ab5f-25d0699e3736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:44.067413 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57772d8-60ff-4486-ab5f-25d0699e3736 admin admin] PATH_INFO: `/v3/projects/6146902809a942a5a517cd3b3dafa0f2/users/eabf68ed556e4732ad6f66affed96dbb/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:44.087491 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57772d8-60ff-4486-ab5f-25d0699e3736 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6146902809a942a5a517cd3b3dafa0f2, user_id=eabf68ed556e4732ad6f66affed96dbb, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:44.091603 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57772d8-60ff-4486-ab5f-25d0699e3736 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:44.178319 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1255/2493] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:40:44 2022] PUT /identity/v3/projects/6146902809a942a5a517cd3b3dafa0f2/users/eabf68ed556e4732ad6f66affed96dbb/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:40:44.188504 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c93b347-3cbf-4830-b06e-3974ad17d756 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:44.188504 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c93b347-3cbf-4830-b06e-3974ad17d756 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:44.188823 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c93b347-3cbf-4830-b06e-3974ad17d756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:44.218502 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-2c93b347-3cbf-4830-b06e-3974ad17d756 None None] MFA Rules not processed for user `eabf68ed556e4732ad6f66affed96dbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:44.219934 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-2c93b347-3cbf-4830-b06e-3974ad17d756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:44.268838 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:44.294088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1239/2494] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:40:44.298688 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57772d8-60ff-4486-ab5f-25d0699e3736 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:44.361258 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57772d8-60ff-4486-ab5f-25d0699e3736 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:44.366463 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad1a779-0888-438f-abaa-e51ac18f4d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:44.367121 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1a779-0888-438f-abaa-e51ac18f4d85 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:44.367397 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1a779-0888-438f-abaa-e51ac18f4d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:44.367737 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1a779-0888-438f-abaa-e51ac18f4d85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:44.368433 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad1a779-0888-438f-abaa-e51ac18f4d85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:44.370335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad1a779-0888-438f-abaa-e51ac18f4d85 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:44.450128 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1256/2495] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:40:44 2022] POST /identity/v3/projects => generated 385 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:40:44.451189 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c93b347-3cbf-4830-b06e-3974ad17d756 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:44.465358 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c93b347-3cbf-4830-b06e-3974ad17d756 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:44.525914 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612c279-16ac-46e0-9385-048e8c0a84ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:44.526585 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612c279-16ac-46e0-9385-048e8c0a84ae admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:44.526875 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612c279-16ac-46e0-9385-048e8c0a84ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:44.527146 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612c279-16ac-46e0-9385-048e8c0a84ae admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:44.527884 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5612c279-16ac-46e0-9385-048e8c0a84ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:44.530117 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5612c279-16ac-46e0-9385-048e8c0a84ae admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:44.644086 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1240/2496] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:40:44 2022] POST /identity/v3/users => generated 293 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:40:44.648559 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad1a779-0888-438f-abaa-e51ac18f4d85 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:44.658789 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad1a779-0888-438f-abaa-e51ac18f4d85 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:44.664106 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3b4b0-6495-4298-8ee0-4fb46e32c540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:44.664748 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3b4b0-6495-4298-8ee0-4fb46e32c540 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:44.665034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3b4b0-6495-4298-8ee0-4fb46e32c540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:44.665313 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3b4b0-6495-4298-8ee0-4fb46e32c540 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:44.666036 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a3b4b0-6495-4298-8ee0-4fb46e32c540 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:44.667662 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a3b4b0-6495-4298-8ee0-4fb46e32c540 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:44.678359 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1257/2497] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:40:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:40:44.682243 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612c279-16ac-46e0-9385-048e8c0a84ae admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:44.694294 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612c279-16ac-46e0-9385-048e8c0a84ae admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:44.697089 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0f12f-542e-4095-a2c0-060375a90c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:44.697738 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0f12f-542e-4095-a2c0-060375a90c21 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:44.698021 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0f12f-542e-4095-a2c0-060375a90c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:44.698298 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0f12f-542e-4095-a2c0-060375a90c21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:44.699090 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c0f12f-542e-4095-a2c0-060375a90c21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:44.700685 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c0f12f-542e-4095-a2c0-060375a90c21 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:44.711268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1241/2498] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:40:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:40:44.715372 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3b4b0-6495-4298-8ee0-4fb46e32c540 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:44.729157 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3b4b0-6495-4298-8ee0-4fb46e32c540 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:44.732022 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0f12f-542e-4095-a2c0-060375a90c21 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:44.733821 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da0689-7ef2-4b70-a21d-349aaf0abe10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:44.734246 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-d5c0f12f-542e-4095-a2c0-060375a90c21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:44.735371 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da0689-7ef2-4b70-a21d-349aaf0abe10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:44.735371 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da0689-7ef2-4b70-a21d-349aaf0abe10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:44.735371 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da0689-7ef2-4b70-a21d-349aaf0abe10 admin admin] PATH_INFO: `/v3/projects/931024f5d2bf4bb5b744516a2156c90d/users/f3413d126b1a476da5c2eac62b2ec95e/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:44.748451 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0f12f-542e-4095-a2c0-060375a90c21 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:44.753477 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392c85b-12c9-4d6f-9634-feec9f4620cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:44.754398 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392c85b-12c9-4d6f-9634-feec9f4620cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:44.754706 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392c85b-12c9-4d6f-9634-feec9f4620cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:44.754915 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da0689-7ef2-4b70-a21d-349aaf0abe10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=931024f5d2bf4bb5b744516a2156c90d, user_id=f3413d126b1a476da5c2eac62b2ec95e, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:44.755041 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392c85b-12c9-4d6f-9634-feec9f4620cc admin admin] PATH_INFO: `/v3/users/2033ad40d45747aa8294e4feb5a6cfe0` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:44.756285 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2392c85b-12c9-4d6f-9634-feec9f4620cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=2033ad40d45747aa8294e4feb5a6cfe0)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:44.757719 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da0689-7ef2-4b70-a21d-349aaf0abe10 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:44.758246 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2392c85b-12c9-4d6f-9634-feec9f4620cc admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:44.871989 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1258/2499] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:40:44 2022] PUT /identity/v3/projects/931024f5d2bf4bb5b744516a2156c90d/users/f3413d126b1a476da5c2eac62b2ec95e/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:40:44.883991 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48050960-e6a8-49d9-887b-3b17de6b4584 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:44.884285 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48050960-e6a8-49d9-887b-3b17de6b4584 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:44.884568 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48050960-e6a8-49d9-887b-3b17de6b4584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:44.913147 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-48050960-e6a8-49d9-887b-3b17de6b4584 None None] MFA Rules not processed for user `f3413d126b1a476da5c2eac62b2ec95e`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:44.917797 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-48050960-e6a8-49d9-887b-3b17de6b4584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:44.924306 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:44.991865 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1259/2500] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:40:44.997381 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48050960-e6a8-49d9-887b-3b17de6b4584 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:45.061517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48050960-e6a8-49d9-887b-3b17de6b4584 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:45.066098 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444c05b-7f5e-49d8-b593-b2de2265400d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:45.066745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444c05b-7f5e-49d8-b593-b2de2265400d admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:45.067523 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444c05b-7f5e-49d8-b593-b2de2265400d admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:45.067846 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444c05b-7f5e-49d8-b593-b2de2265400d admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:45.068514 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444c05b-7f5e-49d8-b593-b2de2265400d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:45.070391 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444c05b-7f5e-49d8-b593-b2de2265400d admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:45.228785 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-2392c85b-12c9-4d6f-9634-feec9f4620cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2033ad40d45747aa8294e4feb5a6cfe0'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:45.236444 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-2392c85b-12c9-4d6f-9634-feec9f4620cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2033ad40d45747aa8294e4feb5a6cfe0'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:45.248679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-2392c85b-12c9-4d6f-9634-feec9f4620cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2033ad40d45747aa8294e4feb5a6cfe0'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:45.249384 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-2392c85b-12c9-4d6f-9634-feec9f4620cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2033ad40d45747aa8294e4feb5a6cfe0'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:45.302993 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1260/2501] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:40:44 2022] POST /identity/v3/projects => generated 387 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:40:45.306738 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444c05b-7f5e-49d8-b593-b2de2265400d admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:45.352250 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-2392c85b-12c9-4d6f-9634-feec9f4620cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2033ad40d45747aa8294e4feb5a6cfe0'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:45.358616 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1242/2502] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:40:44 2022] DELETE /identity/v3/users/2033ad40d45747aa8294e4feb5a6cfe0 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:40:45.368526 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444c05b-7f5e-49d8-b593-b2de2265400d admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:45.425466 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50581b8-2604-4726-98c5-fe1f8b6ba266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:45.426144 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50581b8-2604-4726-98c5-fe1f8b6ba266 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:45.426929 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50581b8-2604-4726-98c5-fe1f8b6ba266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:45.427316 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50581b8-2604-4726-98c5-fe1f8b6ba266 admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:45.428034 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50581b8-2604-4726-98c5-fe1f8b6ba266 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:45.429975 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50581b8-2604-4726-98c5-fe1f8b6ba266 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:45.443436 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:45.532741 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1261/2503] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:40:45 2022] POST /identity/v3/users => generated 300 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:40:45.534113 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392c85b-12c9-4d6f-9634-feec9f4620cc admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:45.535512 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-2392c85b-12c9-4d6f-9634-feec9f4620cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:45.549165 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392c85b-12c9-4d6f-9634-feec9f4620cc admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:45.554624 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcc1e6d-a332-465c-b16e-2207781e2620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:45.555282 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc1e6d-a332-465c-b16e-2207781e2620 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:45.555589 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc1e6d-a332-465c-b16e-2207781e2620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:45.555867 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc1e6d-a332-465c-b16e-2207781e2620 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:45.556620 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcc1e6d-a332-465c-b16e-2207781e2620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:45.558196 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcc1e6d-a332-465c-b16e-2207781e2620 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:45.569102 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1243/2504] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:40:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:40:45.573685 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50581b8-2604-4726-98c5-fe1f8b6ba266 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:45.582888 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50581b8-2604-4726-98c5-fe1f8b6ba266 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:45.587442 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba93754-1288-42e6-92c6-9b6e56c5bd3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:45.588343 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba93754-1288-42e6-92c6-9b6e56c5bd3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:45.588625 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba93754-1288-42e6-92c6-9b6e56c5bd3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:45.588898 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba93754-1288-42e6-92c6-9b6e56c5bd3c admin admin] PATH_INFO: `/v3/projects/ae07ecc9dd494305b3f822ee12bbfed5/users/f0add609c78241178d3adc02eb00220a/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:45.604069 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba93754-1288-42e6-92c6-9b6e56c5bd3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae07ecc9dd494305b3f822ee12bbfed5, user_id=f0add609c78241178d3adc02eb00220a, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:45.606724 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba93754-1288-42e6-92c6-9b6e56c5bd3c admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:45.628349 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcc1e6d-a332-465c-b16e-2207781e2620 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:45.630150 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-8dcc1e6d-a332-465c-b16e-2207781e2620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:45.643284 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcc1e6d-a332-465c-b16e-2207781e2620 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:45.648724 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c83b363-663e-4480-8231-5b31beb568c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:45.652809 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c83b363-663e-4480-8231-5b31beb568c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:45.652809 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c83b363-663e-4480-8231-5b31beb568c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:45.652809 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c83b363-663e-4480-8231-5b31beb568c6 admin admin] PATH_INFO: `/v3/projects/d2e4afda121042939745304ce6f76a8b` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:45.652809 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c83b363-663e-4480-8231-5b31beb568c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2e4afda121042939745304ce6f76a8b)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:45.654048 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c83b363-663e-4480-8231-5b31beb568c6 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:45.672569 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:45.717967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1262/2505] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:40:45 2022] PUT /identity/v3/projects/ae07ecc9dd494305b3f822ee12bbfed5/users/f0add609c78241178d3adc02eb00220a/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:40:45.722160 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba93754-1288-42e6-92c6-9b6e56c5bd3c admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:45.791295 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba93754-1288-42e6-92c6-9b6e56c5bd3c admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:45.796381 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b982e-e8f2-4f79-8742-f10609599294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:45.797046 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b982e-e8f2-4f79-8742-f10609599294 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:45.797359 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b982e-e8f2-4f79-8742-f10609599294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:45.797638 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b982e-e8f2-4f79-8742-f10609599294 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:45.798370 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567b982e-e8f2-4f79-8742-f10609599294 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:45.800044 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567b982e-e8f2-4f79-8742-f10609599294 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:45.814060 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1263/2506] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:40:45 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:40:45.815028 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b982e-e8f2-4f79-8742-f10609599294 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:45.821500 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-2c83b363-663e-4480-8231-5b31beb568c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2e4afda121042939745304ce6f76a8b', 'request_id': 'req-2c83b363-663e-4480-8231-5b31beb568c6'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:45.824831 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b982e-e8f2-4f79-8742-f10609599294 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:45.830669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-2c83b363-663e-4480-8231-5b31beb568c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2e4afda121042939745304ce6f76a8b', 'request_id': 'req-2c83b363-663e-4480-8231-5b31beb568c6'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:45.835193 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-2c83b363-663e-4480-8231-5b31beb568c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2e4afda121042939745304ce6f76a8b', 'request_id': 'req-2c83b363-663e-4480-8231-5b31beb568c6'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:45.883588 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab8c776-bb10-4f7d-837d-c3fc0b36da87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:45.884492 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8c776-bb10-4f7d-837d-c3fc0b36da87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:45.884773 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8c776-bb10-4f7d-837d-c3fc0b36da87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:45.885050 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8c776-bb10-4f7d-837d-c3fc0b36da87 admin admin] PATH_INFO: `/v3/projects/ae07ecc9dd494305b3f822ee12bbfed5/users/f0add609c78241178d3adc02eb00220a/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:45.886995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab8c776-bb10-4f7d-837d-c3fc0b36da87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae07ecc9dd494305b3f822ee12bbfed5, user_id=f0add609c78241178d3adc02eb00220a, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:45.889754 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab8c776-bb10-4f7d-837d-c3fc0b36da87 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:45.900470 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1264/2507] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:40:45 2022] PUT /identity/v3/projects/ae07ecc9dd494305b3f822ee12bbfed5/users/f0add609c78241178d3adc02eb00220a/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:40:45.900824 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-2c83b363-663e-4480-8231-5b31beb568c6 admin admin] The token cache is being invalidate because project d2e4afda121042939745304ce6f76a8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:40:45.901148 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-2c83b363-663e-4480-8231-5b31beb568c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:45.901789 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-2c83b363-663e-4480-8231-5b31beb568c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:45.903351 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1244/2508] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:40:45 2022] DELETE /identity/v3/projects/d2e4afda121042939745304ce6f76a8b => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:40:45.908025 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f81fec-cd69-4963-ae40-5b582fc1a062 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:45.908316 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f81fec-cd69-4963-ae40-5b582fc1a062 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:45.908597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f81fec-cd69-4963-ae40-5b582fc1a062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:45.910517 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e55834-c174-477b-8920-4e92751a4b66 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:45.910812 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e55834-c174-477b-8920-4e92751a4b66 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:45.911135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e55834-c174-477b-8920-4e92751a4b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:45.927527 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-c0f81fec-cd69-4963-ae40-5b582fc1a062 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:45.928806 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-c0f81fec-cd69-4963-ae40-5b582fc1a062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:45.937306 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-13e55834-c174-477b-8920-4e92751a4b66 None None] MFA Rules not processed for user `f0add609c78241178d3adc02eb00220a`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:45.938496 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-13e55834-c174-477b-8920-4e92751a4b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:45.970936 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:45.996788 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1265/2509] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:40:46.005265 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f0e4dc-eecb-4a65-9258-a13aab5a576d None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:46.012080 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f0e4dc-eecb-4a65-9258-a13aab5a576d None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:46.012080 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f0e4dc-eecb-4a65-9258-a13aab5a576d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:46.013080 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1245/2510] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:45 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:40:46.024194 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-d0f0e4dc-eecb-4a65-9258-a13aab5a576d None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:46.024408 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e55834-c174-477b-8920-4e92751a4b66 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:46.025576 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-d0f0e4dc-eecb-4a65-9258-a13aab5a576d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:46.025846 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-13e55834-c174-477b-8920-4e92751a4b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:46.034454 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1266/2511] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:40:46.038655 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f0e4dc-eecb-4a65-9258-a13aab5a576d None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:46.047688 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f0e4dc-eecb-4a65-9258-a13aab5a576d None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:46.052894 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94307fe0-83b0-4c30-8e76-49949387a8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:46.053579 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94307fe0-83b0-4c30-8e76-49949387a8ea admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:46.053862 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94307fe0-83b0-4c30-8e76-49949387a8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:46.057299 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94307fe0-83b0-4c30-8e76-49949387a8ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:46.057299 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94307fe0-83b0-4c30-8e76-49949387a8ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:40:46.057299 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94307fe0-83b0-4c30-8e76-49949387a8ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:46.057299 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94307fe0-83b0-4c30-8e76-49949387a8ea admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:46.076689 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1267/2512] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:40:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:40:46.077535 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94307fe0-83b0-4c30-8e76-49949387a8ea admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:46.086794 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e55834-c174-477b-8920-4e92751a4b66 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:46.087294 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94307fe0-83b0-4c30-8e76-49949387a8ea admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:46.090626 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99646be5-a84d-460d-873f-b9dcc16d2574 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:46.091265 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99646be5-a84d-460d-873f-b9dcc16d2574 service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:46.091541 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99646be5-a84d-460d-873f-b9dcc16d2574 service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:46.091842 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99646be5-a84d-460d-873f-b9dcc16d2574 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:46.092158 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21364bd3-8456-482e-8c00-cfcf5158ed39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:46.092800 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21364bd3-8456-482e-8c00-cfcf5158ed39 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:46.093107 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21364bd3-8456-482e-8c00-cfcf5158ed39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:46.093415 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21364bd3-8456-482e-8c00-cfcf5158ed39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:46.094135 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21364bd3-8456-482e-8c00-cfcf5158ed39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:46.096064 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21364bd3-8456-482e-8c00-cfcf5158ed39 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:46.101752 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99646be5-a84d-460d-873f-b9dcc16d2574 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:46.103166 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99646be5-a84d-460d-873f-b9dcc16d2574 service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:46.113028 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1246/2513] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:40:46 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 15:40:46.205434 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1268/2514] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:40:46 2022] POST /identity/v3/projects => generated 371 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:40:46.210294 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99646be5-a84d-460d-873f-b9dcc16d2574 service cinder] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:46.221746 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99646be5-a84d-460d-873f-b9dcc16d2574 service cinder] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:46.252339 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:46.278778 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff85fc-f1aa-4981-b9e6-104e912e3bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:46.279439 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff85fc-f1aa-4981-b9e6-104e912e3bd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:46.279759 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff85fc-f1aa-4981-b9e6-104e912e3bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:46.280037 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff85fc-f1aa-4981-b9e6-104e912e3bd7 admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:46.280710 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff85fc-f1aa-4981-b9e6-104e912e3bd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:46.282632 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff85fc-f1aa-4981-b9e6-104e912e3bd7 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:46.398194 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1247/2515] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:40:46 2022] POST /identity/v3/users => generated 286 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:40:46.402287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21364bd3-8456-482e-8c00-cfcf5158ed39 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:46.413067 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21364bd3-8456-482e-8c00-cfcf5158ed39 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:46.417706 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5bfd8-9e68-4cef-8fce-5c64339a113f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:46.418356 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5bfd8-9e68-4cef-8fce-5c64339a113f admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:46.418649 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5bfd8-9e68-4cef-8fce-5c64339a113f admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:46.418916 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5bfd8-9e68-4cef-8fce-5c64339a113f admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:46.419662 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df5bfd8-9e68-4cef-8fce-5c64339a113f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:46.421269 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df5bfd8-9e68-4cef-8fce-5c64339a113f admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:46.431770 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1269/2516] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:40:46.435688 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff85fc-f1aa-4981-b9e6-104e912e3bd7 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:46.445846 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff85fc-f1aa-4981-b9e6-104e912e3bd7 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:46.450510 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7828fbd-2f92-4ca3-8516-0c214530267e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:46.451157 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7828fbd-2f92-4ca3-8516-0c214530267e admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:46.451443 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7828fbd-2f92-4ca3-8516-0c214530267e admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:46.451758 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7828fbd-2f92-4ca3-8516-0c214530267e admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:46.452476 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7828fbd-2f92-4ca3-8516-0c214530267e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:46.454122 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7828fbd-2f92-4ca3-8516-0c214530267e admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:46.470798 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1248/2517] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:40:46.473949 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5bfd8-9e68-4cef-8fce-5c64339a113f admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:46.483925 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5bfd8-9e68-4cef-8fce-5c64339a113f admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:46.489585 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e820267c-0061-4c01-bee1-79ace0125084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:46.490590 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e820267c-0061-4c01-bee1-79ace0125084 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:46.490874 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e820267c-0061-4c01-bee1-79ace0125084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:46.491273 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e820267c-0061-4c01-bee1-79ace0125084 admin admin] PATH_INFO: `/v3/projects/74aed37262f04c5b8cd552b2a4edd0ea/users/5fe6848dd346440e97b91dd79610a874/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:46.508115 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e820267c-0061-4c01-bee1-79ace0125084 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74aed37262f04c5b8cd552b2a4edd0ea, user_id=5fe6848dd346440e97b91dd79610a874, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:46.510787 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e820267c-0061-4c01-bee1-79ace0125084 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:46.581446 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1270/2518] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:40:46 2022] PUT /identity/v3/projects/74aed37262f04c5b8cd552b2a4edd0ea/users/5fe6848dd346440e97b91dd79610a874/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:40:46.592511 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f032328b-5bab-4831-8ca0-12cfe53c3424 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:46.592984 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f032328b-5bab-4831-8ca0-12cfe53c3424 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:46.593100 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f032328b-5bab-4831-8ca0-12cfe53c3424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:46.621077 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-f032328b-5bab-4831-8ca0-12cfe53c3424 None None] MFA Rules not processed for user `5fe6848dd346440e97b91dd79610a874`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:46.622288 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-f032328b-5bab-4831-8ca0-12cfe53c3424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:46.686655 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1249/2519] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:40:46.691304 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e820267c-0061-4c01-bee1-79ace0125084 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:46.753123 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e820267c-0061-4c01-bee1-79ace0125084 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:46.757883 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09080cc9-cf6d-459c-8858-7412b7c1caf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:46.758517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09080cc9-cf6d-459c-8858-7412b7c1caf2 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:46.758801 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09080cc9-cf6d-459c-8858-7412b7c1caf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:46.759077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09080cc9-cf6d-459c-8858-7412b7c1caf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:46.759806 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09080cc9-cf6d-459c-8858-7412b7c1caf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:46.761751 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09080cc9-cf6d-459c-8858-7412b7c1caf2 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:46.867950 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1271/2520] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:40:46 2022] POST /identity/v3/projects => generated 373 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:40:46.871945 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f032328b-5bab-4831-8ca0-12cfe53c3424 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:46.882657 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f032328b-5bab-4831-8ca0-12cfe53c3424 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:46.938141 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0dbe2d-8104-472e-b96f-bc72392854db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:46.938802 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0dbe2d-8104-472e-b96f-bc72392854db admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:46.939085 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0dbe2d-8104-472e-b96f-bc72392854db admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:46.939399 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0dbe2d-8104-472e-b96f-bc72392854db admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:46.940103 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0dbe2d-8104-472e-b96f-bc72392854db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:46.941995 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0dbe2d-8104-472e-b96f-bc72392854db admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:47.038921 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1250/2521] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:40:46 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:40:47.042696 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09080cc9-cf6d-459c-8858-7412b7c1caf2 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:47.052871 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09080cc9-cf6d-459c-8858-7412b7c1caf2 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:47.057484 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac003bb-1357-4d71-abe8-0d90817be1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:47.058135 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac003bb-1357-4d71-abe8-0d90817be1c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:47.058415 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac003bb-1357-4d71-abe8-0d90817be1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:47.058690 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac003bb-1357-4d71-abe8-0d90817be1c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:47.059443 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac003bb-1357-4d71-abe8-0d90817be1c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:47.061075 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac003bb-1357-4d71-abe8-0d90817be1c6 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:47.071307 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1272/2522] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:40:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:40:47.075295 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0dbe2d-8104-472e-b96f-bc72392854db admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:47.086242 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0dbe2d-8104-472e-b96f-bc72392854db admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:47.091068 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59670e6-c2c5-4590-a885-727de61d925d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:47.092051 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59670e6-c2c5-4590-a885-727de61d925d admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:47.092331 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59670e6-c2c5-4590-a885-727de61d925d admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:47.092604 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59670e6-c2c5-4590-a885-727de61d925d admin admin] PATH_INFO: `/v3/projects/b69194410fb547a99d528b1fb3b27e73/users/19f27998156e413cac45c16780dafd7a/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:47.109669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59670e6-c2c5-4590-a885-727de61d925d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b69194410fb547a99d528b1fb3b27e73, user_id=19f27998156e413cac45c16780dafd7a, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:47.112739 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59670e6-c2c5-4590-a885-727de61d925d admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:47.170433 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1251/2523] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:40:47 2022] PUT /identity/v3/projects/b69194410fb547a99d528b1fb3b27e73/users/19f27998156e413cac45c16780dafd7a/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:40:47.174540 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac003bb-1357-4d71-abe8-0d90817be1c6 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:47.185190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac003bb-1357-4d71-abe8-0d90817be1c6 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:47.229327 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:47.262447 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4bc850-0924-4102-af76-ee05bfc28ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:47.262447 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4bc850-0924-4102-af76-ee05bfc28ab8 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:47.262447 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4bc850-0924-4102-af76-ee05bfc28ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:47.262447 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4bc850-0924-4102-af76-ee05bfc28ab8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:47.262899 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4bc850-0924-4102-af76-ee05bfc28ab8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:47.264584 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4bc850-0924-4102-af76-ee05bfc28ab8 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:47.276687 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1273/2524] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:40:47 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:40:47.282967 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59670e6-c2c5-4590-a885-727de61d925d admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:47.287626 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4bc850-0924-4102-af76-ee05bfc28ab8 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:47.294820 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-4c4bc850-0924-4102-af76-ee05bfc28ab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:47.294980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59670e6-c2c5-4590-a885-727de61d925d admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:47.299055 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf5f88-20f2-4bb6-acf7-6ab811e3a1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:47.300009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf5f88-20f2-4bb6-acf7-6ab811e3a1c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:47.300283 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf5f88-20f2-4bb6-acf7-6ab811e3a1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:47.300583 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf5f88-20f2-4bb6-acf7-6ab811e3a1c3 admin admin] PATH_INFO: `/v3/projects/b69194410fb547a99d528b1fb3b27e73/users/19f27998156e413cac45c16780dafd7a/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:47.303248 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edf5f88-20f2-4bb6-acf7-6ab811e3a1c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b69194410fb547a99d528b1fb3b27e73, user_id=19f27998156e413cac45c16780dafd7a, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:47.305942 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edf5f88-20f2-4bb6-acf7-6ab811e3a1c3 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:47.308329 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:47.316900 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1252/2525] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:40:47 2022] PUT /identity/v3/projects/b69194410fb547a99d528b1fb3b27e73/users/19f27998156e413cac45c16780dafd7a/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:40:47.327321 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1b684-679b-40e1-b88b-ebc57963958d None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:47.327643 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1b684-679b-40e1-b88b-ebc57963958d None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:47.327926 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1b684-679b-40e1-b88b-ebc57963958d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:47.356774 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4bc850-0924-4102-af76-ee05bfc28ab8 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:47.358444 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-95d1b684-679b-40e1-b88b-ebc57963958d None None] MFA Rules not processed for user `19f27998156e413cac45c16780dafd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:47.359724 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-95d1b684-679b-40e1-b88b-ebc57963958d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:47.409247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87922bd-08b0-4b81-9344-5d163d69730c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:47.409848 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87922bd-08b0-4b81-9344-5d163d69730c service cinder] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:47.410047 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87922bd-08b0-4b81-9344-5d163d69730c service cinder] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:47.410384 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87922bd-08b0-4b81-9344-5d163d69730c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:47.412847 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-b87922bd-08b0-4b81-9344-5d163d69730c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:47.433404 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1253/2526] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:47 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:40:47.447929 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1b684-679b-40e1-b88b-ebc57963958d None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:47.458129 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1b684-679b-40e1-b88b-ebc57963958d None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:47.461874 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ebac4c-7a2f-4417-96c5-4e27ec3338b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:47.462633 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ebac4c-7a2f-4417-96c5-4e27ec3338b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:47.462924 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ebac4c-7a2f-4417-96c5-4e27ec3338b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:47.463232 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ebac4c-7a2f-4417-96c5-4e27ec3338b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:47.473093 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ebac4c-7a2f-4417-96c5-4e27ec3338b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:47.475243 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ebac4c-7a2f-4417-96c5-4e27ec3338b0 service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:47.475429 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87922bd-08b0-4b81-9344-5d163d69730c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:47.476902 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87922bd-08b0-4b81-9344-5d163d69730c service cinder] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:47.485401 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1254/2527] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:40:47 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 15:40:47.485926 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1274/2528] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:40:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 15:40:47.787536 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ebac4c-7a2f-4417-96c5-4e27ec3338b0 service cinder] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:47.796270 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ebac4c-7a2f-4417-96c5-4e27ec3338b0 service cinder] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:47.799918 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36908699-67ff-4023-8e05-24187da9dc8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:47.800547 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36908699-67ff-4023-8e05-24187da9dc8d service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:47.800825 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36908699-67ff-4023-8e05-24187da9dc8d service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:47.801099 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36908699-67ff-4023-8e05-24187da9dc8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:47.857077 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36908699-67ff-4023-8e05-24187da9dc8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:47.858493 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36908699-67ff-4023-8e05-24187da9dc8d service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:47.867660 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1255/2529] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:40:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:40:51.452650 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87922bd-08b0-4b81-9344-5d163d69730c service cinder] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:51.454079 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-b87922bd-08b0-4b81-9344-5d163d69730c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:51.503448 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:51.522618 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87922bd-08b0-4b81-9344-5d163d69730c service cinder] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:51.527093 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727b17e-98c2-49e7-bc1e-ae498d61c9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:51.528003 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727b17e-98c2-49e7-bc1e-ae498d61c9b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:51.528882 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727b17e-98c2-49e7-bc1e-ae498d61c9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:51.528882 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727b17e-98c2-49e7-bc1e-ae498d61c9b5 admin admin] PATH_INFO: `/v3/users/282ba7270e784402af6f0763fb6263e8` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:51.529772 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6727b17e-98c2-49e7-bc1e-ae498d61c9b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=282ba7270e784402af6f0763fb6263e8)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:51.531721 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6727b17e-98c2-49e7-bc1e-ae498d61c9b5 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:51.685470 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-6727b17e-98c2-49e7-bc1e-ae498d61c9b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '282ba7270e784402af6f0763fb6263e8'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:51.686133 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-6727b17e-98c2-49e7-bc1e-ae498d61c9b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '282ba7270e784402af6f0763fb6263e8'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:51.693479 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-6727b17e-98c2-49e7-bc1e-ae498d61c9b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '282ba7270e784402af6f0763fb6263e8'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:51.694219 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-6727b17e-98c2-49e7-bc1e-ae498d61c9b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '282ba7270e784402af6f0763fb6263e8'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:51.707460 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-6727b17e-98c2-49e7-bc1e-ae498d61c9b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '282ba7270e784402af6f0763fb6263e8'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:51.815614 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1275/2530] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:40:51 2022] DELETE /identity/v3/users/282ba7270e784402af6f0763fb6263e8 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:40:51.820811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca11a3-e1ac-4e37-8a12-db0d13062ae5 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:51.821264 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca11a3-e1ac-4e37-8a12-db0d13062ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:51.821378 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca11a3-e1ac-4e37-8a12-db0d13062ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:51.839458 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-6cca11a3-e1ac-4e37-8a12-db0d13062ae5 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:51.841812 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-6cca11a3-e1ac-4e37-8a12-db0d13062ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:51.901205 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1256/2531] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:40:51.911315 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727b17e-98c2-49e7-bc1e-ae498d61c9b5 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:51.912771 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-6727b17e-98c2-49e7-bc1e-ae498d61c9b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:51.970885 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727b17e-98c2-49e7-bc1e-ae498d61c9b5 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:51.974606 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69ad85b-e96e-4852-b0bf-196af2050fbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:51.975250 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ad85b-e96e-4852-b0bf-196af2050fbb service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:51.975531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ad85b-e96e-4852-b0bf-196af2050fbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:51.975844 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ad85b-e96e-4852-b0bf-196af2050fbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:51.984698 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69ad85b-e96e-4852-b0bf-196af2050fbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:51.986371 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69ad85b-e96e-4852-b0bf-196af2050fbb service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:51.996230 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1276/2532] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:40:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:40:52.334671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca11a3-e1ac-4e37-8a12-db0d13062ae5 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:52.334671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-6cca11a3-e1ac-4e37-8a12-db0d13062ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:52.340304 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:52.390217 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca11a3-e1ac-4e37-8a12-db0d13062ae5 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:52.393736 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c4228e-ceda-44e6-b075-99473203328f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:52.394403 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4228e-ceda-44e6-b075-99473203328f service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:52.394681 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4228e-ceda-44e6-b075-99473203328f service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:52.394993 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4228e-ceda-44e6-b075-99473203328f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:52.397292 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-16c4228e-ceda-44e6-b075-99473203328f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:52.453988 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c4228e-ceda-44e6-b075-99473203328f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:52.455523 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c4228e-ceda-44e6-b075-99473203328f service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:52.456539 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69ad85b-e96e-4852-b0bf-196af2050fbb service neutron] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:52.458612 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-b69ad85b-e96e-4852-b0bf-196af2050fbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:52.469051 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1257/2533] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:40:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:40:52.473367 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69ad85b-e96e-4852-b0bf-196af2050fbb service neutron] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:52.479685 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ee496e-aa11-4d1a-8ab2-bad697790d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:52.480577 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee496e-aa11-4d1a-8ab2-bad697790d34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:52.480856 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee496e-aa11-4d1a-8ab2-bad697790d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:52.481132 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee496e-aa11-4d1a-8ab2-bad697790d34 admin admin] PATH_INFO: `/v3/projects/2f68e6b7c9dc49dc86e3296517db6b50` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:52.482710 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ee496e-aa11-4d1a-8ab2-bad697790d34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f68e6b7c9dc49dc86e3296517db6b50)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:52.485006 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ee496e-aa11-4d1a-8ab2-bad697790d34 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:52.609726 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-46ee496e-aa11-4d1a-8ab2-bad697790d34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f68e6b7c9dc49dc86e3296517db6b50', 'request_id': 'req-46ee496e-aa11-4d1a-8ab2-bad697790d34'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:52.615662 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-46ee496e-aa11-4d1a-8ab2-bad697790d34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f68e6b7c9dc49dc86e3296517db6b50', 'request_id': 'req-46ee496e-aa11-4d1a-8ab2-bad697790d34'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:52.617591 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:52.622120 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-46ee496e-aa11-4d1a-8ab2-bad697790d34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f68e6b7c9dc49dc86e3296517db6b50', 'request_id': 'req-46ee496e-aa11-4d1a-8ab2-bad697790d34'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:52.763681 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-46ee496e-aa11-4d1a-8ab2-bad697790d34 admin admin] The token cache is being invalidate because project 2f68e6b7c9dc49dc86e3296517db6b50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:40:52.763979 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-46ee496e-aa11-4d1a-8ab2-bad697790d34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:52.764607 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-46ee496e-aa11-4d1a-8ab2-bad697790d34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:52.766162 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1277/2534] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:40:52 2022] DELETE /identity/v3/projects/2f68e6b7c9dc49dc86e3296517db6b50 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:40:52.770221 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c4228e-ceda-44e6-b075-99473203328f service cinder] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:52.781274 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c4228e-ceda-44e6-b075-99473203328f service cinder] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:52.838668 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ea059c-b234-4a73-8f4d-2bbf19c959e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:52.839533 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea059c-b234-4a73-8f4d-2bbf19c959e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:52.839833 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea059c-b234-4a73-8f4d-2bbf19c959e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:52.840105 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea059c-b234-4a73-8f4d-2bbf19c959e3 admin admin] PATH_INFO: `/v3/users/d6ee81fbd7d54838b7845e16e9ab985a` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:52.841277 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ea059c-b234-4a73-8f4d-2bbf19c959e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6ee81fbd7d54838b7845e16e9ab985a)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:52.843254 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ea059c-b234-4a73-8f4d-2bbf19c959e3 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:53.018200 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-76ea059c-b234-4a73-8f4d-2bbf19c959e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6ee81fbd7d54838b7845e16e9ab985a'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:53.025197 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-76ea059c-b234-4a73-8f4d-2bbf19c959e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6ee81fbd7d54838b7845e16e9ab985a'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:53.029572 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:53.037425 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-76ea059c-b234-4a73-8f4d-2bbf19c959e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6ee81fbd7d54838b7845e16e9ab985a'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:53.038079 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-76ea059c-b234-4a73-8f4d-2bbf19c959e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6ee81fbd7d54838b7845e16e9ab985a'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:53.117632 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-76ea059c-b234-4a73-8f4d-2bbf19c959e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6ee81fbd7d54838b7845e16e9ab985a'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:53.123971 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1258/2535] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:40:52 2022] DELETE /identity/v3/users/d6ee81fbd7d54838b7845e16e9ab985a => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:40:53.542211 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ee496e-aa11-4d1a-8ab2-bad697790d34 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:53.544962 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-46ee496e-aa11-4d1a-8ab2-bad697790d34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:53.558220 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ee496e-aa11-4d1a-8ab2-bad697790d34 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:53.616949 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ef9eb-1a06-4b37-8210-7a5d24171df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:53.617803 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ef9eb-1a06-4b37-8210-7a5d24171df2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:53.618079 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ef9eb-1a06-4b37-8210-7a5d24171df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:53.618348 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ef9eb-1a06-4b37-8210-7a5d24171df2 admin admin] PATH_INFO: `/v3/projects/6e273d007934466caa5503738d5c59b1` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:53.619505 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89ef9eb-1a06-4b37-8210-7a5d24171df2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e273d007934466caa5503738d5c59b1)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:53.621518 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89ef9eb-1a06-4b37-8210-7a5d24171df2 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:53.724185 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-a89ef9eb-1a06-4b37-8210-7a5d24171df2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e273d007934466caa5503738d5c59b1', 'request_id': 'req-a89ef9eb-1a06-4b37-8210-7a5d24171df2'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:53.730461 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-a89ef9eb-1a06-4b37-8210-7a5d24171df2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e273d007934466caa5503738d5c59b1', 'request_id': 'req-a89ef9eb-1a06-4b37-8210-7a5d24171df2'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:53.736937 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-a89ef9eb-1a06-4b37-8210-7a5d24171df2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e273d007934466caa5503738d5c59b1', 'request_id': 'req-a89ef9eb-1a06-4b37-8210-7a5d24171df2'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:53.786322 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-a89ef9eb-1a06-4b37-8210-7a5d24171df2 admin admin] The token cache is being invalidate because project 6e273d007934466caa5503738d5c59b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:40:53.786612 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-a89ef9eb-1a06-4b37-8210-7a5d24171df2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:53.787941 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-a89ef9eb-1a06-4b37-8210-7a5d24171df2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:53.789480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1278/2536] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:40:53 2022] DELETE /identity/v3/projects/6e273d007934466caa5503738d5c59b1 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:40:53.794422 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5c9dd-63b0-47f5-b5be-065f71d7501d None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:53.794716 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5c9dd-63b0-47f5-b5be-065f71d7501d None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:53.795026 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5c9dd-63b0-47f5-b5be-065f71d7501d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:53.813171 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-c0f5c9dd-63b0-47f5-b5be-065f71d7501d None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:53.814428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-c0f5c9dd-63b0-47f5-b5be-065f71d7501d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:53.879629 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1259/2537] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:40:53.884312 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147f10e-d5ac-4652-89fa-47a9b9d138d3 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:53.884601 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147f10e-d5ac-4652-89fa-47a9b9d138d3 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:53.884880 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147f10e-d5ac-4652-89fa-47a9b9d138d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:53.903648 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-f147f10e-d5ac-4652-89fa-47a9b9d138d3 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:53.904842 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-f147f10e-d5ac-4652-89fa-47a9b9d138d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:53.913231 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1279/2538] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:40:53.917381 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5c9dd-63b0-47f5-b5be-065f71d7501d None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:53.927191 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5c9dd-63b0-47f5-b5be-065f71d7501d None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:53.931875 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4625c-3007-4330-a717-8714411d613e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:53.932529 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4625c-3007-4330-a717-8714411d613e admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:53.932819 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4625c-3007-4330-a717-8714411d613e admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:53.933097 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4625c-3007-4330-a717-8714411d613e admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:53.933869 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4625c-3007-4330-a717-8714411d613e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:40:53.934196 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4625c-3007-4330-a717-8714411d613e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:53.935807 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4625c-3007-4330-a717-8714411d613e admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:53.948472 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1260/2539] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:40:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:40:53.952882 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147f10e-d5ac-4652-89fa-47a9b9d138d3 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:53.976653 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147f10e-d5ac-4652-89fa-47a9b9d138d3 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:53.977233 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff34864-4427-4439-ac90-8e7c843cf688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:53.977797 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff34864-4427-4439-ac90-8e7c843cf688 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:53.978269 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff34864-4427-4439-ac90-8e7c843cf688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:53.978379 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff34864-4427-4439-ac90-8e7c843cf688 admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:53.979013 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff34864-4427-4439-ac90-8e7c843cf688 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:53.980921 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff34864-4427-4439-ac90-8e7c843cf688 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:54.091320 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1280/2540] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:40:53 2022] POST /identity/v3/projects => generated 379 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:40:54.094551 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4625c-3007-4330-a717-8714411d613e admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:54.104624 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4625c-3007-4330-a717-8714411d613e admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:54.161529 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd6511-7c4c-4eb0-bf9d-f65d47b623b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:54.162185 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd6511-7c4c-4eb0-bf9d-f65d47b623b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:54.162492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd6511-7c4c-4eb0-bf9d-f65d47b623b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:54.162765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd6511-7c4c-4eb0-bf9d-f65d47b623b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:54.163887 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cd6511-7c4c-4eb0-bf9d-f65d47b623b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:54.165370 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cd6511-7c4c-4eb0-bf9d-f65d47b623b9 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:54.256916 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1261/2541] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:40:54 2022] POST /identity/v3/users => generated 290 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:40:54.262361 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff34864-4427-4439-ac90-8e7c843cf688 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:54.272739 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff34864-4427-4439-ac90-8e7c843cf688 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:54.277738 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b2a063-de2b-4813-84f2-6011985538dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:54.286364 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2a063-de2b-4813-84f2-6011985538dd admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:54.286364 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2a063-de2b-4813-84f2-6011985538dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:54.286364 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2a063-de2b-4813-84f2-6011985538dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:54.286364 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b2a063-de2b-4813-84f2-6011985538dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:54.286364 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b2a063-de2b-4813-84f2-6011985538dd admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:54.293702 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1281/2542] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:40:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:40:54.296583 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd6511-7c4c-4eb0-bf9d-f65d47b623b9 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:54.307343 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd6511-7c4c-4eb0-bf9d-f65d47b623b9 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:54.312153 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8197ecb-c74f-420b-a8db-611e2dd82fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:54.312815 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8197ecb-c74f-420b-a8db-611e2dd82fbf admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:54.313103 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8197ecb-c74f-420b-a8db-611e2dd82fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:54.313405 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8197ecb-c74f-420b-a8db-611e2dd82fbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:54.314136 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8197ecb-c74f-420b-a8db-611e2dd82fbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:54.315765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8197ecb-c74f-420b-a8db-611e2dd82fbf admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:54.326764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1262/2543] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:40:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:40:54.330883 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b2a063-de2b-4813-84f2-6011985538dd admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:54.344804 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b2a063-de2b-4813-84f2-6011985538dd admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:54.349423 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755d884e-64f3-4959-8828-c88e455d0e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:54.350290 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d884e-64f3-4959-8828-c88e455d0e6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:54.350567 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d884e-64f3-4959-8828-c88e455d0e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:54.350836 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d884e-64f3-4959-8828-c88e455d0e6f admin admin] PATH_INFO: `/v3/projects/45c6a486f87e42299232005611384900/users/33ad04b47e9342eb96c9df7261591c04/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:54.365974 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755d884e-64f3-4959-8828-c88e455d0e6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=45c6a486f87e42299232005611384900, user_id=33ad04b47e9342eb96c9df7261591c04, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:54.368630 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755d884e-64f3-4959-8828-c88e455d0e6f admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:54.449224 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1282/2544] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:40:54 2022] PUT /identity/v3/projects/45c6a486f87e42299232005611384900/users/33ad04b47e9342eb96c9df7261591c04/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:40:54.461019 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb318cb-0626-4c88-b7e4-75f5ec618a3a None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:54.461309 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb318cb-0626-4c88-b7e4-75f5ec618a3a None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:54.461586 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb318cb-0626-4c88-b7e4-75f5ec618a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:54.493970 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-aeb318cb-0626-4c88-b7e4-75f5ec618a3a None None] MFA Rules not processed for user `33ad04b47e9342eb96c9df7261591c04`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:54.493970 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-aeb318cb-0626-4c88-b7e4-75f5ec618a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:54.522347 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:54.553391 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1263/2545] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:40:54.558052 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755d884e-64f3-4959-8828-c88e455d0e6f admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:54.576906 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:54.621378 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755d884e-64f3-4959-8828-c88e455d0e6f admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:54.626065 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd5ae2-9311-4718-bc3e-e08efc4c676f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:54.626729 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd5ae2-9311-4718-bc3e-e08efc4c676f admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:54.627018 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd5ae2-9311-4718-bc3e-e08efc4c676f admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:54.627298 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd5ae2-9311-4718-bc3e-e08efc4c676f admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:54.628037 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bd5ae2-9311-4718-bc3e-e08efc4c676f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:54.631827 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bd5ae2-9311-4718-bc3e-e08efc4c676f admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:54.764494 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1283/2546] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:40:54 2022] POST /identity/v3/projects => generated 379 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:40:54.768447 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb318cb-0626-4c88-b7e4-75f5ec618a3a None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:54.781310 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb318cb-0626-4c88-b7e4-75f5ec618a3a None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:54.838003 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b32f0a-4d3b-4b9b-b12f-ad15de7da08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:54.838721 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b32f0a-4d3b-4b9b-b12f-ad15de7da08f admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:54.839000 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b32f0a-4d3b-4b9b-b12f-ad15de7da08f admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:54.839272 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b32f0a-4d3b-4b9b-b12f-ad15de7da08f admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:54.839994 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b32f0a-4d3b-4b9b-b12f-ad15de7da08f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:54.841877 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b32f0a-4d3b-4b9b-b12f-ad15de7da08f admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:54.947147 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1264/2547] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:40:54 2022] POST /identity/v3/users => generated 296 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:40:54.951085 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd5ae2-9311-4718-bc3e-e08efc4c676f admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:54.961533 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd5ae2-9311-4718-bc3e-e08efc4c676f admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:54.966202 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a207de2c-5e63-43e3-a4ae-24ade207b21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:54.966869 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207de2c-5e63-43e3-a4ae-24ade207b21f admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:54.967152 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207de2c-5e63-43e3-a4ae-24ade207b21f admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:54.967430 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207de2c-5e63-43e3-a4ae-24ade207b21f admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:54.968208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a207de2c-5e63-43e3-a4ae-24ade207b21f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:54.969900 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a207de2c-5e63-43e3-a4ae-24ade207b21f admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:54.980519 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1284/2548] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:40:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:40:54.984907 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b32f0a-4d3b-4b9b-b12f-ad15de7da08f admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:54.996471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b32f0a-4d3b-4b9b-b12f-ad15de7da08f admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:55.001770 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b65fe7-f72f-4db3-9403-d68f45b52310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:55.002690 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b65fe7-f72f-4db3-9403-d68f45b52310 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:55.002985 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b65fe7-f72f-4db3-9403-d68f45b52310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:55.003308 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b65fe7-f72f-4db3-9403-d68f45b52310 admin admin] PATH_INFO: `/v3/projects/519e90394aa944819dd318cbe38ad61c/users/adb3d610e50b46aa936b9224948da051/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:55.019848 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b65fe7-f72f-4db3-9403-d68f45b52310 admin admin] RBAC: Authorizing `identity:create_grant(project_id=519e90394aa944819dd318cbe38ad61c, user_id=adb3d610e50b46aa936b9224948da051, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:55.022496 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b65fe7-f72f-4db3-9403-d68f45b52310 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:55.118847 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1265/2549] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:40:54 2022] PUT /identity/v3/projects/519e90394aa944819dd318cbe38ad61c/users/adb3d610e50b46aa936b9224948da051/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:40:55.123041 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a207de2c-5e63-43e3-a4ae-24ade207b21f admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:55.134815 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a207de2c-5e63-43e3-a4ae-24ade207b21f admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:55.191005 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba6c6d5-8137-47c6-b245-e6962e154303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:55.191724 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba6c6d5-8137-47c6-b245-e6962e154303 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:55.192011 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba6c6d5-8137-47c6-b245-e6962e154303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:55.192288 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba6c6d5-8137-47c6-b245-e6962e154303 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:55.193098 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba6c6d5-8137-47c6-b245-e6962e154303 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:55.195288 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba6c6d5-8137-47c6-b245-e6962e154303 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:55.204960 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1285/2550] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:40:55 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:40:55.209051 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b65fe7-f72f-4db3-9403-d68f45b52310 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:55.219659 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b65fe7-f72f-4db3-9403-d68f45b52310 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:55.224591 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d481c36-c82b-45a0-b50f-f469021f1276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:55.225561 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d481c36-c82b-45a0-b50f-f469021f1276 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:55.225937 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d481c36-c82b-45a0-b50f-f469021f1276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:55.226209 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d481c36-c82b-45a0-b50f-f469021f1276 admin admin] PATH_INFO: `/v3/projects/519e90394aa944819dd318cbe38ad61c/users/adb3d610e50b46aa936b9224948da051/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:55.228205 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d481c36-c82b-45a0-b50f-f469021f1276 admin admin] RBAC: Authorizing `identity:create_grant(project_id=519e90394aa944819dd318cbe38ad61c, user_id=adb3d610e50b46aa936b9224948da051, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:55.230954 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d481c36-c82b-45a0-b50f-f469021f1276 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:55.242419 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1266/2551] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:40:55 2022] PUT /identity/v3/projects/519e90394aa944819dd318cbe38ad61c/users/adb3d610e50b46aa936b9224948da051/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:40:55.252763 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bbf64-c826-4f0a-9648-a0dc373d2d4a None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:55.253058 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bbf64-c826-4f0a-9648-a0dc373d2d4a None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:55.253344 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bbf64-c826-4f0a-9648-a0dc373d2d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:55.282222 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-f64bbf64-c826-4f0a-9648-a0dc373d2d4a None None] MFA Rules not processed for user `adb3d610e50b46aa936b9224948da051`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:55.283363 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-f64bbf64-c826-4f0a-9648-a0dc373d2d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:55.352132 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1286/2552] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:55 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:40:55.645709 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d481c36-c82b-45a0-b50f-f469021f1276 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:55.647484 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-6d481c36-c82b-45a0-b50f-f469021f1276 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:55.706516 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d481c36-c82b-45a0-b50f-f469021f1276 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:55.710954 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c38f4f-946e-44e1-aca1-90c98baa50e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:55.711628 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c38f4f-946e-44e1-aca1-90c98baa50e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:55.711915 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c38f4f-946e-44e1-aca1-90c98baa50e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:55.712197 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c38f4f-946e-44e1-aca1-90c98baa50e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:55.722785 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c38f4f-946e-44e1-aca1-90c98baa50e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:55.724864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c38f4f-946e-44e1-aca1-90c98baa50e1 service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:55.739984 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1267/2553] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:40:55 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:40:56.805343 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64bbf64-c826-4f0a-9648-a0dc373d2d4a None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:56.814462 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64bbf64-c826-4f0a-9648-a0dc373d2d4a None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:56.819662 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d0dbba-aabf-4efd-8967-a2217ba63c56 tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adb3d610e50b46aa936b9224948da051', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '519e90394aa944819dd318cbe38ad61c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:56.820322 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d0dbba-aabf-4efd-8967-a2217ba63c56 tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:56.820582 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d0dbba-aabf-4efd-8967-a2217ba63c56 tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:56.820858 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d0dbba-aabf-4efd-8967-a2217ba63c56 tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:56.821594 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d0dbba-aabf-4efd-8967-a2217ba63c56 tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:40:56.821912 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d0dbba-aabf-4efd-8967-a2217ba63c56 tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:56.823485 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d0dbba-aabf-4efd-8967-a2217ba63c56 tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:56.830210 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:56.835650 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1287/2554] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:40:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:40:56.839807 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c38f4f-946e-44e1-aca1-90c98baa50e1 service cinder] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:56.848934 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c38f4f-946e-44e1-aca1-90c98baa50e1 service cinder] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:56.854023 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dcbce5-6f84-43f8-86ab-a51dc388c9b0 tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adb3d610e50b46aa936b9224948da051', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '519e90394aa944819dd318cbe38ad61c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:56.854668 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dcbce5-6f84-43f8-86ab-a51dc388c9b0 tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:56.854954 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dcbce5-6f84-43f8-86ab-a51dc388c9b0 tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:56.855235 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dcbce5-6f84-43f8-86ab-a51dc388c9b0 tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:56.855965 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dcbce5-6f84-43f8-86ab-a51dc388c9b0 tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:56.857835 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dcbce5-6f84-43f8-86ab-a51dc388c9b0 tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:56.988095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1268/2555] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:40:56 2022] POST /identity/v3/projects => generated 354 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:40:56.992975 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d0dbba-aabf-4efd-8967-a2217ba63c56 tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:57.003761 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d0dbba-aabf-4efd-8967-a2217ba63c56 tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:57.059470 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd504ba1-4f5a-42b2-8af4-5686c2aa46f1 tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adb3d610e50b46aa936b9224948da051', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '519e90394aa944819dd318cbe38ad61c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:57.060086 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd504ba1-4f5a-42b2-8af4-5686c2aa46f1 tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:57.060370 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd504ba1-4f5a-42b2-8af4-5686c2aa46f1 tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:57.060650 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd504ba1-4f5a-42b2-8af4-5686c2aa46f1 tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:57.061372 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd504ba1-4f5a-42b2-8af4-5686c2aa46f1 tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:40:57.061683 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd504ba1-4f5a-42b2-8af4-5686c2aa46f1 tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:57.063273 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd504ba1-4f5a-42b2-8af4-5686c2aa46f1 tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:57.074314 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1288/2556] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:40:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:40:57.109973 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dcbce5-6f84-43f8-86ab-a51dc388c9b0 tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:57.121124 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dcbce5-6f84-43f8-86ab-a51dc388c9b0 tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:57.125648 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1887696f-850c-486c-a8ce-d981a51e388c tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adb3d610e50b46aa936b9224948da051', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '519e90394aa944819dd318cbe38ad61c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:57.126608 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887696f-850c-486c-a8ce-d981a51e388c tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:57.126893 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887696f-850c-486c-a8ce-d981a51e388c tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:57.127170 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887696f-850c-486c-a8ce-d981a51e388c tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] PATH_INFO: `/v3/projects/f7fc1b7f75cb45429c3787678ef2d519` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:57.128386 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1887696f-850c-486c-a8ce-d981a51e388c tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f7fc1b7f75cb45429c3787678ef2d519)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:57.130305 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1887696f-850c-486c-a8ce-d981a51e388c tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:57.131775 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:57.236888 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-1887696f-850c-486c-a8ce-d981a51e388c tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7fc1b7f75cb45429c3787678ef2d519', 'request_id': 'req-1887696f-850c-486c-a8ce-d981a51e388c'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:57.243195 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-1887696f-850c-486c-a8ce-d981a51e388c tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7fc1b7f75cb45429c3787678ef2d519', 'request_id': 'req-1887696f-850c-486c-a8ce-d981a51e388c'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:57.249764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-1887696f-850c-486c-a8ce-d981a51e388c tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7fc1b7f75cb45429c3787678ef2d519', 'request_id': 'req-1887696f-850c-486c-a8ce-d981a51e388c'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:57.310177 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-1887696f-850c-486c-a8ce-d981a51e388c tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] The token cache is being invalidate because project f7fc1b7f75cb45429c3787678ef2d519 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:40:57.310392 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-1887696f-850c-486c-a8ce-d981a51e388c tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:57.311047 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-1887696f-850c-486c-a8ce-d981a51e388c tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:57.312623 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1269/2557] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:40:57 2022] DELETE /identity/v3/projects/f7fc1b7f75cb45429c3787678ef2d519 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:40:57.955443 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd504ba1-4f5a-42b2-8af4-5686c2aa46f1 tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:57.955443 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-dd504ba1-4f5a-42b2-8af4-5686c2aa46f1 tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:58.025058 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd504ba1-4f5a-42b2-8af4-5686c2aa46f1 tempest-VolumeQuotaClassesTest-959925490 tempest-VolumeQuotaClassesTest-959925490-project-admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:58.080222 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0f5924-17f9-49ff-a5c5-5129e02990b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:58.081083 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f5924-17f9-49ff-a5c5-5129e02990b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:58.081362 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f5924-17f9-49ff-a5c5-5129e02990b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:58.081631 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f5924-17f9-49ff-a5c5-5129e02990b4 admin admin] PATH_INFO: `/v3/users/33ad04b47e9342eb96c9df7261591c04` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:58.082912 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0f5924-17f9-49ff-a5c5-5129e02990b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33ad04b47e9342eb96c9df7261591c04)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:58.085430 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0f5924-17f9-49ff-a5c5-5129e02990b4 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:58.280755 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:58.285625 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-6e0f5924-17f9-49ff-a5c5-5129e02990b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33ad04b47e9342eb96c9df7261591c04'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:58.286276 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-6e0f5924-17f9-49ff-a5c5-5129e02990b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33ad04b47e9342eb96c9df7261591c04'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:58.293749 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-6e0f5924-17f9-49ff-a5c5-5129e02990b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33ad04b47e9342eb96c9df7261591c04'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:58.294381 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-6e0f5924-17f9-49ff-a5c5-5129e02990b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33ad04b47e9342eb96c9df7261591c04'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:58.305895 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-6e0f5924-17f9-49ff-a5c5-5129e02990b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33ad04b47e9342eb96c9df7261591c04'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:58.369454 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1289/2558] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:40:57 2022] DELETE /identity/v3/users/33ad04b47e9342eb96c9df7261591c04 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:40:58.373944 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf34192-35a4-4482-9b67-13e5ac967ba6 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:58.374401 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf34192-35a4-4482-9b67-13e5ac967ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:58.374500 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf34192-35a4-4482-9b67-13e5ac967ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:58.392400 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-ddf34192-35a4-4482-9b67-13e5ac967ba6 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:58.393609 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-ddf34192-35a4-4482-9b67-13e5ac967ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:58.453472 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1270/2559] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:40:58.463873 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0f5924-17f9-49ff-a5c5-5129e02990b4 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:58.465271 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-6e0f5924-17f9-49ff-a5c5-5129e02990b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:58.523833 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0f5924-17f9-49ff-a5c5-5129e02990b4 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:58.527546 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951bcb73-617e-4724-8d77-0c4459e9f06d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:58.528227 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951bcb73-617e-4724-8d77-0c4459e9f06d service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:58.528511 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951bcb73-617e-4724-8d77-0c4459e9f06d service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:58.528787 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951bcb73-617e-4724-8d77-0c4459e9f06d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:58.537763 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951bcb73-617e-4724-8d77-0c4459e9f06d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:58.539185 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951bcb73-617e-4724-8d77-0c4459e9f06d service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:58.549263 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1290/2560] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:40:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:40:58.988463 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf34192-35a4-4482-9b67-13e5ac967ba6 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:58.990142 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-ddf34192-35a4-4482-9b67-13e5ac967ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:59.003353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf34192-35a4-4482-9b67-13e5ac967ba6 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:59.010122 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf879faf-872d-412f-89e0-65ef2f5a5893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:59.013924 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf879faf-872d-412f-89e0-65ef2f5a5893 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:59.013924 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf879faf-872d-412f-89e0-65ef2f5a5893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:59.013924 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf879faf-872d-412f-89e0-65ef2f5a5893 admin admin] PATH_INFO: `/v3/projects/45c6a486f87e42299232005611384900` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:59.013924 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf879faf-872d-412f-89e0-65ef2f5a5893 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45c6a486f87e42299232005611384900)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:59.015213 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf879faf-872d-412f-89e0-65ef2f5a5893 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:59.140606 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-cf879faf-872d-412f-89e0-65ef2f5a5893 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45c6a486f87e42299232005611384900', 'request_id': 'req-cf879faf-872d-412f-89e0-65ef2f5a5893'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:59.148167 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-cf879faf-872d-412f-89e0-65ef2f5a5893 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45c6a486f87e42299232005611384900', 'request_id': 'req-cf879faf-872d-412f-89e0-65ef2f5a5893'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:59.159501 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-cf879faf-872d-412f-89e0-65ef2f5a5893 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45c6a486f87e42299232005611384900', 'request_id': 'req-cf879faf-872d-412f-89e0-65ef2f5a5893'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:59.232592 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951bcb73-617e-4724-8d77-0c4459e9f06d service neutron] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:59.233057 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-cf879faf-872d-412f-89e0-65ef2f5a5893 admin admin] The token cache is being invalidate because project 45c6a486f87e42299232005611384900 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:40:59.233057 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-cf879faf-872d-412f-89e0-65ef2f5a5893 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:59.236063 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-951bcb73-617e-4724-8d77-0c4459e9f06d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:59.236173 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-cf879faf-872d-412f-89e0-65ef2f5a5893 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:59.236522 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1271/2561] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:40:58 2022] DELETE /identity/v3/projects/45c6a486f87e42299232005611384900 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:40:59.239992 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf879faf-872d-412f-89e0-65ef2f5a5893 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:59.244554 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-cf879faf-872d-412f-89e0-65ef2f5a5893 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:59.253022 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951bcb73-617e-4724-8d77-0c4459e9f06d service neutron] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:59.280427 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:59.320641 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34f9f4-7553-4000-8766-5bbf830b553c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:59.321764 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34f9f4-7553-4000-8766-5bbf830b553c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:59.323673 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34f9f4-7553-4000-8766-5bbf830b553c admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:59.323673 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34f9f4-7553-4000-8766-5bbf830b553c admin admin] PATH_INFO: `/v3/users/eabf68ed556e4732ad6f66affed96dbb` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:59.323899 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34f9f4-7553-4000-8766-5bbf830b553c admin admin] RBAC: Authorizing `identity:delete_user(user_id=eabf68ed556e4732ad6f66affed96dbb)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:59.326032 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34f9f4-7553-4000-8766-5bbf830b553c admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:59.345452 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf879faf-872d-412f-89e0-65ef2f5a5893 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:59.351147 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1231cf0b-b78c-48a7-80fd-dc72eae0445b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:59.352075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231cf0b-b78c-48a7-80fd-dc72eae0445b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:59.352382 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231cf0b-b78c-48a7-80fd-dc72eae0445b admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:59.352758 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231cf0b-b78c-48a7-80fd-dc72eae0445b admin admin] PATH_INFO: `/v3/users/adb3d610e50b46aa936b9224948da051` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:59.353987 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1231cf0b-b78c-48a7-80fd-dc72eae0445b admin admin] RBAC: Authorizing `identity:delete_user(user_id=adb3d610e50b46aa936b9224948da051)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:59.356022 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1231cf0b-b78c-48a7-80fd-dc72eae0445b admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:59.565958 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-eb34f9f4-7553-4000-8766-5bbf830b553c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eabf68ed556e4732ad6f66affed96dbb'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:59.566558 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-eb34f9f4-7553-4000-8766-5bbf830b553c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eabf68ed556e4732ad6f66affed96dbb'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:59.573443 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-eb34f9f4-7553-4000-8766-5bbf830b553c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eabf68ed556e4732ad6f66affed96dbb'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:59.574099 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-eb34f9f4-7553-4000-8766-5bbf830b553c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eabf68ed556e4732ad6f66affed96dbb'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:59.579719 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:59.590040 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-eb34f9f4-7553-4000-8766-5bbf830b553c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eabf68ed556e4732ad6f66affed96dbb'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:59.595639 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-1231cf0b-b78c-48a7-80fd-dc72eae0445b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adb3d610e50b46aa936b9224948da051'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:59.603101 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-1231cf0b-b78c-48a7-80fd-dc72eae0445b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adb3d610e50b46aa936b9224948da051'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:59.615531 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-1231cf0b-b78c-48a7-80fd-dc72eae0445b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adb3d610e50b46aa936b9224948da051'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:59.616238 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-1231cf0b-b78c-48a7-80fd-dc72eae0445b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adb3d610e50b46aa936b9224948da051'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:59.730082 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1291/2562] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:40:59 2022] DELETE /identity/v3/users/eabf68ed556e4732ad6f66affed96dbb => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:40:59.734619 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ec090-23a3-42f8-bc43-9e18a2584357 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:59.734907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ec090-23a3-42f8-bc43-9e18a2584357 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:59.735181 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ec090-23a3-42f8-bc43-9e18a2584357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:59.753046 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-585ec090-23a3-42f8-bc43-9e18a2584357 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:40:59.754267 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-585ec090-23a3-42f8-bc43-9e18a2584357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:59.785613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-1231cf0b-b78c-48a7-80fd-dc72eae0445b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adb3d610e50b46aa936b9224948da051'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:40:59.792838 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1272/2563] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:40:59 2022] DELETE /identity/v3/users/adb3d610e50b46aa936b9224948da051 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:40:59.813711 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1292/2564] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:40:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:40:59.827253 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1231cf0b-b78c-48a7-80fd-dc72eae0445b admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:40:59.828753 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-1231cf0b-b78c-48a7-80fd-dc72eae0445b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:40:59.890376 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1231cf0b-b78c-48a7-80fd-dc72eae0445b admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:40:59.893975 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe7916-c8c4-40c9-ad60-38e2a6097112 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:40:59.894615 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe7916-c8c4-40c9-ad60-38e2a6097112 service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:40:59.894901 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe7916-c8c4-40c9-ad60-38e2a6097112 service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:40:59.895175 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe7916-c8c4-40c9-ad60-38e2a6097112 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:40:59.939127 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:40:59.954726 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fe7916-c8c4-40c9-ad60-38e2a6097112 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:40:59.956204 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fe7916-c8c4-40c9-ad60-38e2a6097112 service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:40:59.968549 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1273/2565] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:40:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:41:00.267723 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585ec090-23a3-42f8-bc43-9e18a2584357 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:00.269307 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-585ec090-23a3-42f8-bc43-9e18a2584357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:00.282252 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585ec090-23a3-42f8-bc43-9e18a2584357 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:00.288126 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65349d8d-591c-461c-ac14-1c28bba8aa75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:00.289035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65349d8d-591c-461c-ac14-1c28bba8aa75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:00.289316 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65349d8d-591c-461c-ac14-1c28bba8aa75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:00.289600 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65349d8d-591c-461c-ac14-1c28bba8aa75 admin admin] PATH_INFO: `/v3/projects/519e90394aa944819dd318cbe38ad61c` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:00.290823 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65349d8d-591c-461c-ac14-1c28bba8aa75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=519e90394aa944819dd318cbe38ad61c)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:00.292810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65349d8d-591c-461c-ac14-1c28bba8aa75 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:00.409085 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-65349d8d-591c-461c-ac14-1c28bba8aa75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '519e90394aa944819dd318cbe38ad61c', 'request_id': 'req-65349d8d-591c-461c-ac14-1c28bba8aa75'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:00.414838 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-65349d8d-591c-461c-ac14-1c28bba8aa75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '519e90394aa944819dd318cbe38ad61c', 'request_id': 'req-65349d8d-591c-461c-ac14-1c28bba8aa75'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:00.421327 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-65349d8d-591c-461c-ac14-1c28bba8aa75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '519e90394aa944819dd318cbe38ad61c', 'request_id': 'req-65349d8d-591c-461c-ac14-1c28bba8aa75'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:00.457365 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe7916-c8c4-40c9-ad60-38e2a6097112 service neutron] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:00.458836 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-67fe7916-c8c4-40c9-ad60-38e2a6097112 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:00.483268 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe7916-c8c4-40c9-ad60-38e2a6097112 service neutron] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:00.507622 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-65349d8d-591c-461c-ac14-1c28bba8aa75 admin admin] The token cache is being invalidate because project 519e90394aa944819dd318cbe38ad61c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:41:00.507919 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-65349d8d-591c-461c-ac14-1c28bba8aa75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:00.508562 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-65349d8d-591c-461c-ac14-1c28bba8aa75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:00.513813 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1293/2566] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:41:00 2022] DELETE /identity/v3/projects/519e90394aa944819dd318cbe38ad61c => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:00.515169 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882d1bc-1e03-4f5e-ae0b-f4c3f9443ed8 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:00.515459 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882d1bc-1e03-4f5e-ae0b-f4c3f9443ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:00.515760 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882d1bc-1e03-4f5e-ae0b-f4c3f9443ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:00.533635 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-4882d1bc-1e03-4f5e-ae0b-f4c3f9443ed8 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:00.534851 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-4882d1bc-1e03-4f5e-ae0b-f4c3f9443ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:00.554039 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00b513-a0da-45dc-ade6-75b9abe8a733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:00.554894 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00b513-a0da-45dc-ade6-75b9abe8a733 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:00.555173 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00b513-a0da-45dc-ade6-75b9abe8a733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:00.555443 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00b513-a0da-45dc-ade6-75b9abe8a733 admin admin] PATH_INFO: `/v3/projects/6146902809a942a5a517cd3b3dafa0f2` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:00.556784 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e00b513-a0da-45dc-ade6-75b9abe8a733 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6146902809a942a5a517cd3b3dafa0f2)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:00.558705 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e00b513-a0da-45dc-ade6-75b9abe8a733 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:00.595056 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1294/2567] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:00.604283 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db6e6c-066d-408b-8c9d-b2c08382484b None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:00.604575 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db6e6c-066d-408b-8c9d-b2c08382484b None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:00.604859 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db6e6c-066d-408b-8c9d-b2c08382484b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:00.612279 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:00.623326 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-93db6e6c-066d-408b-8c9d-b2c08382484b None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:00.624578 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-93db6e6c-066d-408b-8c9d-b2c08382484b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:00.632471 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1295/2568] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:41:00.636552 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db6e6c-066d-408b-8c9d-b2c08382484b None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:00.645248 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db6e6c-066d-408b-8c9d-b2c08382484b None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:00.649832 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e7ddb-c628-4278-a118-5cb6152dfc53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:00.650459 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e7ddb-c628-4278-a118-5cb6152dfc53 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:00.650745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e7ddb-c628-4278-a118-5cb6152dfc53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:00.651022 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e7ddb-c628-4278-a118-5cb6152dfc53 admin admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:00.651779 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e7ddb-c628-4278-a118-5cb6152dfc53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:41:00.652094 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e7ddb-c628-4278-a118-5cb6152dfc53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:00.653692 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e7ddb-c628-4278-a118-5cb6152dfc53 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:00.665750 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1296/2569] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:41:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:41:00.669742 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e7ddb-c628-4278-a118-5cb6152dfc53 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:00.672145 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-6e00b513-a0da-45dc-ade6-75b9abe8a733 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6146902809a942a5a517cd3b3dafa0f2', 'request_id': 'req-6e00b513-a0da-45dc-ade6-75b9abe8a733'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:00.678921 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-6e00b513-a0da-45dc-ade6-75b9abe8a733 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6146902809a942a5a517cd3b3dafa0f2', 'request_id': 'req-6e00b513-a0da-45dc-ade6-75b9abe8a733'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:00.685058 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e7ddb-c628-4278-a118-5cb6152dfc53 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:00.689523 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-6e00b513-a0da-45dc-ade6-75b9abe8a733 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6146902809a942a5a517cd3b3dafa0f2', 'request_id': 'req-6e00b513-a0da-45dc-ade6-75b9abe8a733'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:00.744700 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a28dc-5c40-4dcd-8d5e-dd251ea991f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:00.745405 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a28dc-5c40-4dcd-8d5e-dd251ea991f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:00.745692 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a28dc-5c40-4dcd-8d5e-dd251ea991f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:00.745971 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a28dc-5c40-4dcd-8d5e-dd251ea991f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:00.746692 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a28dc-5c40-4dcd-8d5e-dd251ea991f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:00.748913 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a28dc-5c40-4dcd-8d5e-dd251ea991f0 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:00.791998 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-6e00b513-a0da-45dc-ade6-75b9abe8a733 admin admin] The token cache is being invalidate because project 6146902809a942a5a517cd3b3dafa0f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:41:00.792289 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-6e00b513-a0da-45dc-ade6-75b9abe8a733 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:00.792910 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-6e00b513-a0da-45dc-ade6-75b9abe8a733 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:00.794473 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1274/2570] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:41:00 2022] DELETE /identity/v3/projects/6146902809a942a5a517cd3b3dafa0f2 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:00.798644 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00b513-a0da-45dc-ade6-75b9abe8a733 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:00.800088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-6e00b513-a0da-45dc-ade6-75b9abe8a733 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:00.813146 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00b513-a0da-45dc-ade6-75b9abe8a733 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:00.820922 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc092f0-9dd2-44db-acb2-636aba0e0f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:00.820922 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc092f0-9dd2-44db-acb2-636aba0e0f84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:00.820922 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc092f0-9dd2-44db-acb2-636aba0e0f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:00.820922 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc092f0-9dd2-44db-acb2-636aba0e0f84 admin admin] PATH_INFO: `/v3/users/f3413d126b1a476da5c2eac62b2ec95e` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:00.821603 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc092f0-9dd2-44db-acb2-636aba0e0f84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3413d126b1a476da5c2eac62b2ec95e)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:00.822677 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc092f0-9dd2-44db-acb2-636aba0e0f84 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:00.868916 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1297/2571] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:41:00 2022] POST /identity/v3/projects => generated 387 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:41:00.873185 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a28dc-5c40-4dcd-8d5e-dd251ea991f0 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:00.874944 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-ff6a28dc-5c40-4dcd-8d5e-dd251ea991f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:00.940986 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a28dc-5c40-4dcd-8d5e-dd251ea991f0 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:00.945649 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc297751-4609-4323-adc8-72c791605750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:00.946312 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc297751-4609-4323-adc8-72c791605750 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:00.946591 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc297751-4609-4323-adc8-72c791605750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:00.946867 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc297751-4609-4323-adc8-72c791605750 admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:00.947630 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc297751-4609-4323-adc8-72c791605750 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:00.949612 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc297751-4609-4323-adc8-72c791605750 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:01.051919 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1298/2572] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:41:00 2022] POST /identity/v3/users => generated 294 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:41:01.053002 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc297751-4609-4323-adc8-72c791605750 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:01.058563 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-dcc092f0-9dd2-44db-acb2-636aba0e0f84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3413d126b1a476da5c2eac62b2ec95e'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:01.066253 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc297751-4609-4323-adc8-72c791605750 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:01.067780 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-dcc092f0-9dd2-44db-acb2-636aba0e0f84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3413d126b1a476da5c2eac62b2ec95e'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:01.072807 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516ee1fa-b825-41e4-8547-c3e086258383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:01.073486 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ee1fa-b825-41e4-8547-c3e086258383 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:01.073777 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ee1fa-b825-41e4-8547-c3e086258383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:01.074054 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ee1fa-b825-41e4-8547-c3e086258383 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:01.074814 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516ee1fa-b825-41e4-8547-c3e086258383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:01.076474 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516ee1fa-b825-41e4-8547-c3e086258383 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:01.080585 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:01.086298 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-dcc092f0-9dd2-44db-acb2-636aba0e0f84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3413d126b1a476da5c2eac62b2ec95e'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:01.087014 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-dcc092f0-9dd2-44db-acb2-636aba0e0f84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3413d126b1a476da5c2eac62b2ec95e'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:01.087494 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1299/2573] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:41:01.091457 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516ee1fa-b825-41e4-8547-c3e086258383 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:01.100911 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516ee1fa-b825-41e4-8547-c3e086258383 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:01.105551 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a47ce57-15f4-4030-af71-2b8aedb6daac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:01.109696 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47ce57-15f4-4030-af71-2b8aedb6daac admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:01.109696 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47ce57-15f4-4030-af71-2b8aedb6daac admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:01.109696 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47ce57-15f4-4030-af71-2b8aedb6daac admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:01.109696 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a47ce57-15f4-4030-af71-2b8aedb6daac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:01.109696 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a47ce57-15f4-4030-af71-2b8aedb6daac admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:01.118753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1300/2574] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:41:01.122625 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a47ce57-15f4-4030-af71-2b8aedb6daac admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:01.131766 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a47ce57-15f4-4030-af71-2b8aedb6daac admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:01.136833 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aaafdc-eec3-4869-bdcb-10620a69e839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:01.137658 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aaafdc-eec3-4869-bdcb-10620a69e839 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:01.138410 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aaafdc-eec3-4869-bdcb-10620a69e839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:01.138410 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aaafdc-eec3-4869-bdcb-10620a69e839 admin admin] PATH_INFO: `/v3/projects/6223601122a846a89bf3dfa003f6fc6b/users/ed1ed81d04d242e5a053993242b528e9/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:01.153208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aaafdc-eec3-4869-bdcb-10620a69e839 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6223601122a846a89bf3dfa003f6fc6b, user_id=ed1ed81d04d242e5a053993242b528e9, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:01.155842 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aaafdc-eec3-4869-bdcb-10620a69e839 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:01.170978 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-dcc092f0-9dd2-44db-acb2-636aba0e0f84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3413d126b1a476da5c2eac62b2ec95e'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:01.178679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1275/2575] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:41:00 2022] DELETE /identity/v3/users/f3413d126b1a476da5c2eac62b2ec95e => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:41:01.261669 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1301/2576] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:41:01 2022] PUT /identity/v3/projects/6223601122a846a89bf3dfa003f6fc6b/users/ed1ed81d04d242e5a053993242b528e9/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:41:01.272063 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f470b5e-0017-4a36-be9c-b84ee70173b3 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:01.272350 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f470b5e-0017-4a36-be9c-b84ee70173b3 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:01.272635 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f470b5e-0017-4a36-be9c-b84ee70173b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:01.300365 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-9f470b5e-0017-4a36-be9c-b84ee70173b3 None None] MFA Rules not processed for user `ed1ed81d04d242e5a053993242b528e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:01.301562 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-9f470b5e-0017-4a36-be9c-b84ee70173b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:01.372925 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1276/2577] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:01.377296 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aaafdc-eec3-4869-bdcb-10620a69e839 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:01.379054 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-61aaafdc-eec3-4869-bdcb-10620a69e839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:01.403962 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:01.456626 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aaafdc-eec3-4869-bdcb-10620a69e839 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:01.461937 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615f847-3c60-4734-9287-e19bec0f467e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:01.462605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615f847-3c60-4734-9287-e19bec0f467e admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:01.462889 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615f847-3c60-4734-9287-e19bec0f467e admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:01.463168 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615f847-3c60-4734-9287-e19bec0f467e admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:01.463909 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2615f847-3c60-4734-9287-e19bec0f467e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:01.465954 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2615f847-3c60-4734-9287-e19bec0f467e admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:01.645920 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1302/2578] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:41:01 2022] POST /identity/v3/projects => generated 387 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:41:01.649177 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f470b5e-0017-4a36-be9c-b84ee70173b3 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:01.659697 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f470b5e-0017-4a36-be9c-b84ee70173b3 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:01.692033 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615f847-3c60-4734-9287-e19bec0f467e admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:01.695444 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-2615f847-3c60-4734-9287-e19bec0f467e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:01.728230 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76201686-c5e1-487f-bc15-436897e8fa89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:01.729330 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76201686-c5e1-487f-bc15-436897e8fa89 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:01.729616 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76201686-c5e1-487f-bc15-436897e8fa89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:01.729896 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76201686-c5e1-487f-bc15-436897e8fa89 admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:01.730991 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76201686-c5e1-487f-bc15-436897e8fa89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:01.733129 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76201686-c5e1-487f-bc15-436897e8fa89 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:01.767787 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615f847-3c60-4734-9287-e19bec0f467e admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:01.773579 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608caced-6495-4230-884d-ad919ca140e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:01.774287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608caced-6495-4230-884d-ad919ca140e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:01.774568 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608caced-6495-4230-884d-ad919ca140e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:01.774840 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608caced-6495-4230-884d-ad919ca140e5 admin admin] PATH_INFO: `/v3/projects/931024f5d2bf4bb5b744516a2156c90d` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:01.776064 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608caced-6495-4230-884d-ad919ca140e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=931024f5d2bf4bb5b744516a2156c90d)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:01.777980 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608caced-6495-4230-884d-ad919ca140e5 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:01.824816 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1277/2579] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:41:01 2022] POST /identity/v3/users => generated 300 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:41:01.826163 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76201686-c5e1-487f-bc15-436897e8fa89 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:01.836043 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76201686-c5e1-487f-bc15-436897e8fa89 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:01.840666 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ad7dca-40fc-4754-9e87-f9c135680a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:01.842527 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad7dca-40fc-4754-9e87-f9c135680a8f admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:01.842527 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad7dca-40fc-4754-9e87-f9c135680a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:01.842527 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad7dca-40fc-4754-9e87-f9c135680a8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:01.842883 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ad7dca-40fc-4754-9e87-f9c135680a8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:01.844184 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ad7dca-40fc-4754-9e87-f9c135680a8f admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:01.854597 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1278/2580] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:41:01.858431 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ad7dca-40fc-4754-9e87-f9c135680a8f admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:01.868724 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ad7dca-40fc-4754-9e87-f9c135680a8f admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:01.873280 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d01a1-54e2-467a-bd1b-0c4d792781af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:01.874105 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d01a1-54e2-467a-bd1b-0c4d792781af admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:01.874386 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d01a1-54e2-467a-bd1b-0c4d792781af admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:01.878135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d01a1-54e2-467a-bd1b-0c4d792781af admin admin] PATH_INFO: `/v3/projects/c460ed815b4443708c2e78f07ffc322c/users/04799383a493477b9183f521c2ff07b3/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:01.889101 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030d01a1-54e2-467a-bd1b-0c4d792781af admin admin] RBAC: Authorizing `identity:create_grant(project_id=c460ed815b4443708c2e78f07ffc322c, user_id=04799383a493477b9183f521c2ff07b3, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:01.891751 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030d01a1-54e2-467a-bd1b-0c4d792781af admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:01.910014 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-608caced-6495-4230-884d-ad919ca140e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '931024f5d2bf4bb5b744516a2156c90d', 'request_id': 'req-608caced-6495-4230-884d-ad919ca140e5'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:01.917509 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-608caced-6495-4230-884d-ad919ca140e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '931024f5d2bf4bb5b744516a2156c90d', 'request_id': 'req-608caced-6495-4230-884d-ad919ca140e5'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:01.923917 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-608caced-6495-4230-884d-ad919ca140e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '931024f5d2bf4bb5b744516a2156c90d', 'request_id': 'req-608caced-6495-4230-884d-ad919ca140e5'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:01.975393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1279/2581] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:41:01 2022] PUT /identity/v3/projects/c460ed815b4443708c2e78f07ffc322c/users/04799383a493477b9183f521c2ff07b3/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:41:01.979922 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d01a1-54e2-467a-bd1b-0c4d792781af admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:01.998442 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-608caced-6495-4230-884d-ad919ca140e5 admin admin] The token cache is being invalidate because project 931024f5d2bf4bb5b744516a2156c90d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:41:01.999350 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-608caced-6495-4230-884d-ad919ca140e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:01.999873 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-608caced-6495-4230-884d-ad919ca140e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:02.002235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1303/2582] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:41:01 2022] DELETE /identity/v3/projects/931024f5d2bf4bb5b744516a2156c90d => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:02.006055 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608caced-6495-4230-884d-ad919ca140e5 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:02.008020 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-608caced-6495-4230-884d-ad919ca140e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:02.020881 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:02.040862 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:02.053376 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d01a1-54e2-467a-bd1b-0c4d792781af admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:02.059244 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157777e5-0c26-4a58-947d-26aa25a25c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:02.059986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157777e5-0c26-4a58-947d-26aa25a25c56 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:02.060270 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157777e5-0c26-4a58-947d-26aa25a25c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:02.060543 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157777e5-0c26-4a58-947d-26aa25a25c56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:02.061263 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157777e5-0c26-4a58-947d-26aa25a25c56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:02.062905 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157777e5-0c26-4a58-947d-26aa25a25c56 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:02.073430 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1280/2583] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:41:02.074456 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608caced-6495-4230-884d-ad919ca140e5 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:02.077437 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157777e5-0c26-4a58-947d-26aa25a25c56 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:02.079039 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-157777e5-0c26-4a58-947d-26aa25a25c56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:02.079385 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda31a2c-1793-4628-82d0-034cdba39fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:02.080313 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda31a2c-1793-4628-82d0-034cdba39fc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:02.080595 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda31a2c-1793-4628-82d0-034cdba39fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:02.080867 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda31a2c-1793-4628-82d0-034cdba39fc9 admin admin] PATH_INFO: `/v3/users/f0add609c78241178d3adc02eb00220a` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:02.082105 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda31a2c-1793-4628-82d0-034cdba39fc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0add609c78241178d3adc02eb00220a)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:02.084053 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda31a2c-1793-4628-82d0-034cdba39fc9 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:02.092576 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157777e5-0c26-4a58-947d-26aa25a25c56 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:02.103386 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b1ea02-94ff-44c1-b2bf-75651a35bae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:02.108198 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b1ea02-94ff-44c1-b2bf-75651a35bae1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:02.108198 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b1ea02-94ff-44c1-b2bf-75651a35bae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:02.108198 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b1ea02-94ff-44c1-b2bf-75651a35bae1 admin admin] PATH_INFO: `/v3/projects/c460ed815b4443708c2e78f07ffc322c/users/04799383a493477b9183f521c2ff07b3/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:02.108198 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b1ea02-94ff-44c1-b2bf-75651a35bae1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c460ed815b4443708c2e78f07ffc322c, user_id=04799383a493477b9183f521c2ff07b3, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:02.110890 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b1ea02-94ff-44c1-b2bf-75651a35bae1 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:02.122294 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1281/2584] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:41:02 2022] PUT /identity/v3/projects/c460ed815b4443708c2e78f07ffc322c/users/04799383a493477b9183f521c2ff07b3/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:41:02.133076 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa5563-0e9c-443d-8407-d90be69a6389 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:02.133413 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa5563-0e9c-443d-8407-d90be69a6389 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:02.133693 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa5563-0e9c-443d-8407-d90be69a6389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:02.164482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-a3aa5563-0e9c-443d-8407-d90be69a6389 None None] MFA Rules not processed for user `04799383a493477b9183f521c2ff07b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:02.164482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-a3aa5563-0e9c-443d-8407-d90be69a6389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:02.230602 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1282/2585] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:02 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:41:02.242014 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aa5563-0e9c-443d-8407-d90be69a6389 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:02.243471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-a3aa5563-0e9c-443d-8407-d90be69a6389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:02.278300 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-bda31a2c-1793-4628-82d0-034cdba39fc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0add609c78241178d3adc02eb00220a'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:02.279001 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-bda31a2c-1793-4628-82d0-034cdba39fc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0add609c78241178d3adc02eb00220a'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:02.286514 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-bda31a2c-1793-4628-82d0-034cdba39fc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0add609c78241178d3adc02eb00220a'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:02.287406 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-bda31a2c-1793-4628-82d0-034cdba39fc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0add609c78241178d3adc02eb00220a'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:02.303053 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-bda31a2c-1793-4628-82d0-034cdba39fc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0add609c78241178d3adc02eb00220a'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:02.303413 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aa5563-0e9c-443d-8407-d90be69a6389 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:02.308150 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b01ea29-fc6e-443d-95ab-36f9fcc355c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:02.308984 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01ea29-fc6e-443d-95ab-36f9fcc355c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:02.313765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01ea29-fc6e-443d-95ab-36f9fcc355c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:02.313765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01ea29-fc6e-443d-95ab-36f9fcc355c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:02.313765 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-7b01ea29-fc6e-443d-95ab-36f9fcc355c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:02.323764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b01ea29-fc6e-443d-95ab-36f9fcc355c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:02.325200 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b01ea29-fc6e-443d-95ab-36f9fcc355c1 service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:02.334777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1283/2586] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:41:02 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:41:02.386178 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1304/2587] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:41:02 2022] DELETE /identity/v3/users/f0add609c78241178d3adc02eb00220a => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:41:02.480439 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b01ea29-fc6e-443d-95ab-36f9fcc355c1 service cinder] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:02.482093 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-7b01ea29-fc6e-443d-95ab-36f9fcc355c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:02.526652 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:02.548019 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b01ea29-fc6e-443d-95ab-36f9fcc355c1 service cinder] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:02.604727 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6ac02e-080f-4004-aa5c-7d58c023fb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:02.605586 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ac02e-080f-4004-aa5c-7d58c023fb10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:02.605862 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ac02e-080f-4004-aa5c-7d58c023fb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:02.606131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ac02e-080f-4004-aa5c-7d58c023fb10 admin admin] PATH_INFO: `/v3/users/ed1ed81d04d242e5a053993242b528e9` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:02.607335 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6ac02e-080f-4004-aa5c-7d58c023fb10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed1ed81d04d242e5a053993242b528e9)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:02.609270 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6ac02e-080f-4004-aa5c-7d58c023fb10 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:02.834652 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-3c6ac02e-080f-4004-aa5c-7d58c023fb10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed1ed81d04d242e5a053993242b528e9'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:02.842468 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-3c6ac02e-080f-4004-aa5c-7d58c023fb10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed1ed81d04d242e5a053993242b528e9'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:02.855836 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-3c6ac02e-080f-4004-aa5c-7d58c023fb10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed1ed81d04d242e5a053993242b528e9'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:02.858976 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-3c6ac02e-080f-4004-aa5c-7d58c023fb10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed1ed81d04d242e5a053993242b528e9'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:02.859140 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda31a2c-1793-4628-82d0-034cdba39fc9 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:02.859284 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-bda31a2c-1793-4628-82d0-034cdba39fc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:02.872687 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda31a2c-1793-4628-82d0-034cdba39fc9 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:02.877386 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c30588-cf48-4693-b230-cba9610c774f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:02.878253 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c30588-cf48-4693-b230-cba9610c774f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:02.878536 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c30588-cf48-4693-b230-cba9610c774f admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:02.878812 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c30588-cf48-4693-b230-cba9610c774f admin admin] PATH_INFO: `/v3/projects/ae07ecc9dd494305b3f822ee12bbfed5` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:02.880035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c30588-cf48-4693-b230-cba9610c774f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae07ecc9dd494305b3f822ee12bbfed5)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:02.881955 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c30588-cf48-4693-b230-cba9610c774f admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:02.993183 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-3c6ac02e-080f-4004-aa5c-7d58c023fb10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed1ed81d04d242e5a053993242b528e9'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:03.001706 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1284/2588] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:41:02 2022] DELETE /identity/v3/users/ed1ed81d04d242e5a053993242b528e9 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:03.006441 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717bd81-ce4c-4ee3-81ba-3f5ebd5d92fe None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:03.007038 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717bd81-ce4c-4ee3-81ba-3f5ebd5d92fe None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:03.007953 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717bd81-ce4c-4ee3-81ba-3f5ebd5d92fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:03.011934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-67c30588-cf48-4693-b230-cba9610c774f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae07ecc9dd494305b3f822ee12bbfed5', 'request_id': 'req-67c30588-cf48-4693-b230-cba9610c774f'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:03.017889 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-67c30588-cf48-4693-b230-cba9610c774f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae07ecc9dd494305b3f822ee12bbfed5', 'request_id': 'req-67c30588-cf48-4693-b230-cba9610c774f'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:03.024497 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-67c30588-cf48-4693-b230-cba9610c774f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae07ecc9dd494305b3f822ee12bbfed5', 'request_id': 'req-67c30588-cf48-4693-b230-cba9610c774f'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:03.026802 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-9717bd81-ce4c-4ee3-81ba-3f5ebd5d92fe None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:03.028122 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-9717bd81-ce4c-4ee3-81ba-3f5ebd5d92fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:03.087252 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1285/2589] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:41:03.096845 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9717bd81-ce4c-4ee3-81ba-3f5ebd5d92fe None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:03.098355 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-9717bd81-ce4c-4ee3-81ba-3f5ebd5d92fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:03.101569 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-67c30588-cf48-4693-b230-cba9610c774f admin admin] The token cache is being invalidate because project ae07ecc9dd494305b3f822ee12bbfed5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:41:03.105536 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-67c30588-cf48-4693-b230-cba9610c774f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:03.105536 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-67c30588-cf48-4693-b230-cba9610c774f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:03.105536 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1305/2590] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:41:02 2022] DELETE /identity/v3/projects/ae07ecc9dd494305b3f822ee12bbfed5 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:03.110065 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b71354-fdfe-4321-931a-29d5a71c948c None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:03.110258 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:03.110443 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b71354-fdfe-4321-931a-29d5a71c948c None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:03.110863 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b71354-fdfe-4321-931a-29d5a71c948c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:03.133085 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-53b71354-fdfe-4321-931a-29d5a71c948c None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:03.134378 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-53b71354-fdfe-4321-931a-29d5a71c948c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:03.143120 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1306/2591] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:41:03.152209 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811e4e5-80f2-4f70-a616-7c2c4e5ad033 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:03.152502 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811e4e5-80f2-4f70-a616-7c2c4e5ad033 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:03.152785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811e4e5-80f2-4f70-a616-7c2c4e5ad033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:03.165530 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9717bd81-ce4c-4ee3-81ba-3f5ebd5d92fe None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:03.169744 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f02227-cd2d-42be-87bf-2038462ee2b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:03.170403 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f02227-cd2d-42be-87bf-2038462ee2b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:03.170691 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f02227-cd2d-42be-87bf-2038462ee2b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:03.170975 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f02227-cd2d-42be-87bf-2038462ee2b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:03.171624 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-1811e4e5-80f2-4f70-a616-7c2c4e5ad033 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:03.172829 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-1811e4e5-80f2-4f70-a616-7c2c4e5ad033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:03.173295 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-71f02227-cd2d-42be-87bf-2038462ee2b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:03.181265 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1307/2592] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:41:03.185448 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1811e4e5-80f2-4f70-a616-7c2c4e5ad033 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:03.186175 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f02227-cd2d-42be-87bf-2038462ee2b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:03.187633 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f02227-cd2d-42be-87bf-2038462ee2b7 service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:03.194499 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1811e4e5-80f2-4f70-a616-7c2c4e5ad033 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:03.197380 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1286/2593] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:41:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 15:41:03.199536 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c2078c-5a53-4071-931e-3a895bb5e3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:03.200239 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2078c-5a53-4071-931e-3a895bb5e3ce admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:03.200516 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2078c-5a53-4071-931e-3a895bb5e3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:03.200911 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2078c-5a53-4071-931e-3a895bb5e3ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:03.201746 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c2078c-5a53-4071-931e-3a895bb5e3ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:41:03.202169 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c2078c-5a53-4071-931e-3a895bb5e3ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:03.203809 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c2078c-5a53-4071-931e-3a895bb5e3ce admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:03.215644 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1308/2594] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:41:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:41:03.220614 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f02227-cd2d-42be-87bf-2038462ee2b7 service neutron] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:03.233599 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f02227-cd2d-42be-87bf-2038462ee2b7 service neutron] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:03.238613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b31d120-7efc-47be-a654-ddb4700ef099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:03.239269 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31d120-7efc-47be-a654-ddb4700ef099 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:03.239552 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31d120-7efc-47be-a654-ddb4700ef099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:03.239852 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31d120-7efc-47be-a654-ddb4700ef099 admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:03.240532 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b31d120-7efc-47be-a654-ddb4700ef099 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:03.243015 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b31d120-7efc-47be-a654-ddb4700ef099 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:03.349048 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1287/2595] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:41:03 2022] POST /identity/v3/projects => generated 401 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:41:03.353362 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c2078c-5a53-4071-931e-3a895bb5e3ce admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:03.359756 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:03.364050 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c2078c-5a53-4071-931e-3a895bb5e3ce admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:03.430872 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deff968-00fa-49d7-844f-6ffab092d5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:03.432341 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deff968-00fa-49d7-844f-6ffab092d5ba admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:03.432627 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deff968-00fa-49d7-844f-6ffab092d5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:03.432909 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deff968-00fa-49d7-844f-6ffab092d5ba admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:03.433621 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deff968-00fa-49d7-844f-6ffab092d5ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:03.435616 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deff968-00fa-49d7-844f-6ffab092d5ba admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:03.547322 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1309/2596] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:41:03 2022] POST /identity/v3/users => generated 301 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:41:03.551401 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b31d120-7efc-47be-a654-ddb4700ef099 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:03.562678 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b31d120-7efc-47be-a654-ddb4700ef099 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:03.574354 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eef39d-44da-4731-8633-ec683df89bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:03.575099 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eef39d-44da-4731-8633-ec683df89bd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:03.575320 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eef39d-44da-4731-8633-ec683df89bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:03.575612 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eef39d-44da-4731-8633-ec683df89bd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:03.576356 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eef39d-44da-4731-8633-ec683df89bd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:03.578095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eef39d-44da-4731-8633-ec683df89bd6 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:03.592030 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1288/2597] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:03 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:41:03.599980 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deff968-00fa-49d7-844f-6ffab092d5ba admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:03.624460 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deff968-00fa-49d7-844f-6ffab092d5ba admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:03.631080 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc082cd-0e3c-435d-8616-77babafce91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:03.631802 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc082cd-0e3c-435d-8616-77babafce91e admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:03.632263 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc082cd-0e3c-435d-8616-77babafce91e admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:03.632605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc082cd-0e3c-435d-8616-77babafce91e admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:03.633351 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc082cd-0e3c-435d-8616-77babafce91e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:03.635025 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc082cd-0e3c-435d-8616-77babafce91e admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:03.653846 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1310/2598] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:03 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:41:03.653998 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eef39d-44da-4731-8633-ec683df89bd6 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:03.669418 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eef39d-44da-4731-8633-ec683df89bd6 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:03.674771 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c4fcb4-3518-4c9a-ae80-7db572c0edb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:03.675714 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c4fcb4-3518-4c9a-ae80-7db572c0edb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:03.676173 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c4fcb4-3518-4c9a-ae80-7db572c0edb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:03.676486 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c4fcb4-3518-4c9a-ae80-7db572c0edb2 admin admin] PATH_INFO: `/v3/projects/3971a67ffb7e435eb069d7ee573d2cb1/users/86a2b7dd44214043ac8096b07fb71b1b/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:03.693065 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c4fcb4-3518-4c9a-ae80-7db572c0edb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3971a67ffb7e435eb069d7ee573d2cb1, user_id=86a2b7dd44214043ac8096b07fb71b1b, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:03.696569 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c4fcb4-3518-4c9a-ae80-7db572c0edb2 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:03.770600 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1289/2599] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:41:03 2022] PUT /identity/v3/projects/3971a67ffb7e435eb069d7ee573d2cb1/users/86a2b7dd44214043ac8096b07fb71b1b/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:03.782588 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353d7c5-c276-4329-a8d2-57315251ca77 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:03.782881 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353d7c5-c276-4329-a8d2-57315251ca77 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:03.783164 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353d7c5-c276-4329-a8d2-57315251ca77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:03.813108 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-c353d7c5-c276-4329-a8d2-57315251ca77 None None] MFA Rules not processed for user `86a2b7dd44214043ac8096b07fb71b1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:03.814351 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-c353d7c5-c276-4329-a8d2-57315251ca77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:03.881388 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1311/2600] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:03 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:41:03.886473 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c4fcb4-3518-4c9a-ae80-7db572c0edb2 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:03.919769 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353d7c5-c276-4329-a8d2-57315251ca77 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:03.921271 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-c353d7c5-c276-4329-a8d2-57315251ca77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:03.972643 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c4fcb4-3518-4c9a-ae80-7db572c0edb2 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:03.977535 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2830a40a-92dc-44f8-8675-5f34355ccd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:03.978199 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830a40a-92dc-44f8-8675-5f34355ccd23 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:03.978480 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830a40a-92dc-44f8-8675-5f34355ccd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:03.978772 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830a40a-92dc-44f8-8675-5f34355ccd23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:03.979606 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2830a40a-92dc-44f8-8675-5f34355ccd23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:03.981505 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2830a40a-92dc-44f8-8675-5f34355ccd23 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:04.004649 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353d7c5-c276-4329-a8d2-57315251ca77 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:04.010944 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c3f14-7a4d-4eee-ac4f-0780dd364380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:04.011845 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c3f14-7a4d-4eee-ac4f-0780dd364380 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:04.012176 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c3f14-7a4d-4eee-ac4f-0780dd364380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:04.014446 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c3f14-7a4d-4eee-ac4f-0780dd364380 admin admin] PATH_INFO: `/v3/projects/6223601122a846a89bf3dfa003f6fc6b` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:04.014446 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33c3f14-7a4d-4eee-ac4f-0780dd364380 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6223601122a846a89bf3dfa003f6fc6b)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:04.015603 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33c3f14-7a4d-4eee-ac4f-0780dd364380 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:04.032336 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:04.124903 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1290/2601] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:41:03 2022] POST /identity/v3/projects => generated 401 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:41:04.128729 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2830a40a-92dc-44f8-8675-5f34355ccd23 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:04.159086 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-b33c3f14-7a4d-4eee-ac4f-0780dd364380 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6223601122a846a89bf3dfa003f6fc6b', 'request_id': 'req-b33c3f14-7a4d-4eee-ac4f-0780dd364380'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:04.164797 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-b33c3f14-7a4d-4eee-ac4f-0780dd364380 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6223601122a846a89bf3dfa003f6fc6b', 'request_id': 'req-b33c3f14-7a4d-4eee-ac4f-0780dd364380'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:04.171001 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-b33c3f14-7a4d-4eee-ac4f-0780dd364380 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6223601122a846a89bf3dfa003f6fc6b', 'request_id': 'req-b33c3f14-7a4d-4eee-ac4f-0780dd364380'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:04.172278 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:04.192563 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2830a40a-92dc-44f8-8675-5f34355ccd23 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:04.250563 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b8ccb-8683-4918-9f20-131f30522059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:04.251241 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b8ccb-8683-4918-9f20-131f30522059 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:04.251527 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b8ccb-8683-4918-9f20-131f30522059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:04.251837 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b8ccb-8683-4918-9f20-131f30522059 admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:04.252524 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b8ccb-8683-4918-9f20-131f30522059 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:04.254441 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b8ccb-8683-4918-9f20-131f30522059 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:04.261126 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-b33c3f14-7a4d-4eee-ac4f-0780dd364380 admin admin] The token cache is being invalidate because project 6223601122a846a89bf3dfa003f6fc6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:41:04.261795 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-b33c3f14-7a4d-4eee-ac4f-0780dd364380 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:04.262157 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-b33c3f14-7a4d-4eee-ac4f-0780dd364380 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:04.264051 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1312/2602] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:41:03 2022] DELETE /identity/v3/projects/6223601122a846a89bf3dfa003f6fc6b => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:41:04.267551 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c3f14-7a4d-4eee-ac4f-0780dd364380 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:04.268884 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-b33c3f14-7a4d-4eee-ac4f-0780dd364380 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:04.283541 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c3f14-7a4d-4eee-ac4f-0780dd364380 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:04.288463 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3378a3b5-1e06-4254-bcce-ced11127da3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:04.289334 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378a3b5-1e06-4254-bcce-ced11127da3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:04.289638 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378a3b5-1e06-4254-bcce-ced11127da3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:04.289996 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378a3b5-1e06-4254-bcce-ced11127da3b admin admin] PATH_INFO: `/v3/users/04799383a493477b9183f521c2ff07b3` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:04.291213 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3378a3b5-1e06-4254-bcce-ced11127da3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=04799383a493477b9183f521c2ff07b3)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:04.293176 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3378a3b5-1e06-4254-bcce-ced11127da3b admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:04.341109 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1291/2603] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:41:04 2022] POST /identity/v3/users => generated 307 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:41:04.343814 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b8ccb-8683-4918-9f20-131f30522059 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:04.345682 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-587b8ccb-8683-4918-9f20-131f30522059 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:04.364909 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b8ccb-8683-4918-9f20-131f30522059 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:04.370174 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501eb6d-4a9e-45c6-9838-e838db6afe84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:04.371437 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501eb6d-4a9e-45c6-9838-e838db6afe84 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:04.371437 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501eb6d-4a9e-45c6-9838-e838db6afe84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:04.371437 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501eb6d-4a9e-45c6-9838-e838db6afe84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:04.372134 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1501eb6d-4a9e-45c6-9838-e838db6afe84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:04.373789 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1501eb6d-4a9e-45c6-9838-e838db6afe84 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:04.384443 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1292/2604] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:41:04.388586 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501eb6d-4a9e-45c6-9838-e838db6afe84 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:04.398040 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501eb6d-4a9e-45c6-9838-e838db6afe84 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:04.402842 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8c398-f391-498e-bce9-70612ca4fb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:04.403786 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8c398-f391-498e-bce9-70612ca4fb56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:04.404073 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8c398-f391-498e-bce9-70612ca4fb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:04.404354 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8c398-f391-498e-bce9-70612ca4fb56 admin admin] PATH_INFO: `/v3/projects/9d0c8a212f7f4a8f8b49ae71eb5419ae/users/6159498cb23b4087b2292d650547b043/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:04.419030 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db8c398-f391-498e-bce9-70612ca4fb56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d0c8a212f7f4a8f8b49ae71eb5419ae, user_id=6159498cb23b4087b2292d650547b043, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:04.422068 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db8c398-f391-498e-bce9-70612ca4fb56 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:04.490752 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-3378a3b5-1e06-4254-bcce-ced11127da3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04799383a493477b9183f521c2ff07b3'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:04.492032 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-3378a3b5-1e06-4254-bcce-ced11127da3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04799383a493477b9183f521c2ff07b3'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:04.499018 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-3378a3b5-1e06-4254-bcce-ced11127da3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04799383a493477b9183f521c2ff07b3'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:04.499830 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-3378a3b5-1e06-4254-bcce-ced11127da3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04799383a493477b9183f521c2ff07b3'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:04.514863 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-3378a3b5-1e06-4254-bcce-ced11127da3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04799383a493477b9183f521c2ff07b3'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:04.540216 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1293/2605] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:41:04 2022] PUT /identity/v3/projects/9d0c8a212f7f4a8f8b49ae71eb5419ae/users/6159498cb23b4087b2292d650547b043/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:41:04.544064 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8c398-f391-498e-bce9-70612ca4fb56 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:04.545637 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-3db8c398-f391-498e-bce9-70612ca4fb56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:04.608851 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8c398-f391-498e-bce9-70612ca4fb56 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:04.613663 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499d974-0a08-4f04-b624-3814511257ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:04.614316 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499d974-0a08-4f04-b624-3814511257ce admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:04.614616 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499d974-0a08-4f04-b624-3814511257ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:04.614888 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499d974-0a08-4f04-b624-3814511257ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:04.615632 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499d974-0a08-4f04-b624-3814511257ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:04.617301 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499d974-0a08-4f04-b624-3814511257ce admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:04.626942 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1294/2606] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:41:04.630990 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499d974-0a08-4f04-b624-3814511257ce admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:04.637318 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1313/2607] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:41:04 2022] DELETE /identity/v3/users/04799383a493477b9183f521c2ff07b3 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:41:04.641023 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499d974-0a08-4f04-b624-3814511257ce admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:04.704226 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67c8cb-2558-4928-b1cb-30428d303c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:04.705791 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67c8cb-2558-4928-b1cb-30428d303c2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:04.706085 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67c8cb-2558-4928-b1cb-30428d303c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:04.706371 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67c8cb-2558-4928-b1cb-30428d303c2f admin admin] PATH_INFO: `/v3/projects/9d0c8a212f7f4a8f8b49ae71eb5419ae/users/6159498cb23b4087b2292d650547b043/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:04.708584 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff67c8cb-2558-4928-b1cb-30428d303c2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d0c8a212f7f4a8f8b49ae71eb5419ae, user_id=6159498cb23b4087b2292d650547b043, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:04.712122 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff67c8cb-2558-4928-b1cb-30428d303c2f admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:04.725285 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1295/2608] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:41:04 2022] PUT /identity/v3/projects/9d0c8a212f7f4a8f8b49ae71eb5419ae/users/6159498cb23b4087b2292d650547b043/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:41:04.726496 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:04.735622 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85262f19-10c5-48b4-a810-403cb0c0952d None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:04.735922 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85262f19-10c5-48b4-a810-403cb0c0952d None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:04.736205 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85262f19-10c5-48b4-a810-403cb0c0952d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:04.763383 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-85262f19-10c5-48b4-a810-403cb0c0952d None None] MFA Rules not processed for user `6159498cb23b4087b2292d650547b043`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:04.764691 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-85262f19-10c5-48b4-a810-403cb0c0952d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:04.830662 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:04.834526 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1314/2609] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:04 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:04.846288 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67c8cb-2558-4928-b1cb-30428d303c2f admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:04.848259 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-ff67c8cb-2558-4928-b1cb-30428d303c2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:04.910080 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67c8cb-2558-4928-b1cb-30428d303c2f admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:04.914442 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f5b94-9941-4280-ad01-d714a527fde5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:04.915088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f5b94-9941-4280-ad01-d714a527fde5 service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:04.915375 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f5b94-9941-4280-ad01-d714a527fde5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:04.915678 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f5b94-9941-4280-ad01-d714a527fde5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:04.925320 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672f5b94-9941-4280-ad01-d714a527fde5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:04.927898 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672f5b94-9941-4280-ad01-d714a527fde5 service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:04.938401 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1296/2610] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:41:04 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 15:41:05.076926 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85262f19-10c5-48b4-a810-403cb0c0952d None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:05.078358 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-85262f19-10c5-48b4-a810-403cb0c0952d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:05.145468 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85262f19-10c5-48b4-a810-403cb0c0952d None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:05.150631 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b75b57-f2af-4b33-9c3a-875c9f75067f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:05.151486 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b75b57-f2af-4b33-9c3a-875c9f75067f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:05.151801 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b75b57-f2af-4b33-9c3a-875c9f75067f admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:05.152233 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b75b57-f2af-4b33-9c3a-875c9f75067f admin admin] PATH_INFO: `/v3/projects/c460ed815b4443708c2e78f07ffc322c` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:05.153849 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b75b57-f2af-4b33-9c3a-875c9f75067f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c460ed815b4443708c2e78f07ffc322c)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:05.155767 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b75b57-f2af-4b33-9c3a-875c9f75067f admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:05.201331 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f5b94-9941-4280-ad01-d714a527fde5 service cinder] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:05.209916 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f5b94-9941-4280-ad01-d714a527fde5 service cinder] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:05.213585 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f139ba02-a018-4bdf-936e-53b61c10a3b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:05.214249 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139ba02-a018-4bdf-936e-53b61c10a3b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:05.214587 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139ba02-a018-4bdf-936e-53b61c10a3b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:05.214888 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139ba02-a018-4bdf-936e-53b61c10a3b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:05.216690 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-f139ba02-a018-4bdf-936e-53b61c10a3b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:05.278065 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f139ba02-a018-4bdf-936e-53b61c10a3b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:05.279481 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f139ba02-a018-4bdf-936e-53b61c10a3b8 service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:05.283534 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-84b75b57-f2af-4b33-9c3a-875c9f75067f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c460ed815b4443708c2e78f07ffc322c', 'request_id': 'req-84b75b57-f2af-4b33-9c3a-875c9f75067f'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:05.290442 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-84b75b57-f2af-4b33-9c3a-875c9f75067f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c460ed815b4443708c2e78f07ffc322c', 'request_id': 'req-84b75b57-f2af-4b33-9c3a-875c9f75067f'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:05.291223 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1297/2611] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:41:05 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:41:05.302704 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-84b75b57-f2af-4b33-9c3a-875c9f75067f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c460ed815b4443708c2e78f07ffc322c', 'request_id': 'req-84b75b57-f2af-4b33-9c3a-875c9f75067f'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:05.389185 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-84b75b57-f2af-4b33-9c3a-875c9f75067f admin admin] The token cache is being invalidate because project c460ed815b4443708c2e78f07ffc322c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:41:05.390620 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-84b75b57-f2af-4b33-9c3a-875c9f75067f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:05.390620 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-84b75b57-f2af-4b33-9c3a-875c9f75067f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:05.396219 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1315/2612] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:41:05 2022] DELETE /identity/v3/projects/c460ed815b4443708c2e78f07ffc322c => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:05.398457 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10700ade-ee77-4bb3-8367-04331c24e0ce None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:05.398762 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10700ade-ee77-4bb3-8367-04331c24e0ce None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:05.399030 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10700ade-ee77-4bb3-8367-04331c24e0ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:05.421951 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-10700ade-ee77-4bb3-8367-04331c24e0ce None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:05.423271 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-10700ade-ee77-4bb3-8367-04331c24e0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:05.494336 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1298/2613] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:05.502770 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75147eb8-b272-4682-ad75-152009a83338 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:05.503292 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75147eb8-b272-4682-ad75-152009a83338 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:05.503420 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75147eb8-b272-4682-ad75-152009a83338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:05.523470 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-75147eb8-b272-4682-ad75-152009a83338 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:05.524745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-75147eb8-b272-4682-ad75-152009a83338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:05.533438 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1316/2614] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:05.537938 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10700ade-ee77-4bb3-8367-04331c24e0ce None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:05.548998 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10700ade-ee77-4bb3-8367-04331c24e0ce None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:05.554033 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732206a-7042-4e41-b4bd-61c699912532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:05.554698 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732206a-7042-4e41-b4bd-61c699912532 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:05.555238 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732206a-7042-4e41-b4bd-61c699912532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:05.555340 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732206a-7042-4e41-b4bd-61c699912532 admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:05.556053 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b732206a-7042-4e41-b4bd-61c699912532 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:41:05.556361 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b732206a-7042-4e41-b4bd-61c699912532 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:05.557991 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b732206a-7042-4e41-b4bd-61c699912532 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:05.565098 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:05.571011 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1299/2615] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:41:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:41:05.575412 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75147eb8-b272-4682-ad75-152009a83338 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:05.584743 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75147eb8-b272-4682-ad75-152009a83338 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:05.590003 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a69f5-87db-4536-8cce-9dd84a059eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:05.590654 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a69f5-87db-4536-8cce-9dd84a059eac admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:05.590934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a69f5-87db-4536-8cce-9dd84a059eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:05.591207 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a69f5-87db-4536-8cce-9dd84a059eac admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:05.591935 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a69f5-87db-4536-8cce-9dd84a059eac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:05.593817 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a69f5-87db-4536-8cce-9dd84a059eac admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:05.696905 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1317/2616] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:41:05 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:41:05.700989 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732206a-7042-4e41-b4bd-61c699912532 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:05.713339 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732206a-7042-4e41-b4bd-61c699912532 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:05.775088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d606a-c4e6-4ea2-a301-ea8558b5e6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:05.775792 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d606a-c4e6-4ea2-a301-ea8558b5e6f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:05.776077 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d606a-c4e6-4ea2-a301-ea8558b5e6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:05.776379 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d606a-c4e6-4ea2-a301-ea8558b5e6f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:05.777052 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d606a-c4e6-4ea2-a301-ea8558b5e6f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:05.779030 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d606a-c4e6-4ea2-a301-ea8558b5e6f7 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:05.867159 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1300/2617] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:41:05 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:41:05.871283 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a69f5-87db-4536-8cce-9dd84a059eac admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:05.881551 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a69f5-87db-4536-8cce-9dd84a059eac admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:05.886163 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c69b0-e504-4598-8ee8-b6ce886a52d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:05.886816 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c69b0-e504-4598-8ee8-b6ce886a52d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:05.887099 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c69b0-e504-4598-8ee8-b6ce886a52d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:05.887378 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c69b0-e504-4598-8ee8-b6ce886a52d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:05.888148 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2c69b0-e504-4598-8ee8-b6ce886a52d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:05.889741 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2c69b0-e504-4598-8ee8-b6ce886a52d8 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:05.912137 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1318/2618] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:41:05.912301 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d606a-c4e6-4ea2-a301-ea8558b5e6f7 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:05.919194 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d606a-c4e6-4ea2-a301-ea8558b5e6f7 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:05.923987 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c3419-ffe2-41a4-af50-0c596981d913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:05.924645 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c3419-ffe2-41a4-af50-0c596981d913 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:05.924930 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c3419-ffe2-41a4-af50-0c596981d913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:05.925208 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c3419-ffe2-41a4-af50-0c596981d913 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:05.925925 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77c3419-ffe2-41a4-af50-0c596981d913 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:05.927501 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77c3419-ffe2-41a4-af50-0c596981d913 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:05.942493 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1301/2619] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:41:05.943666 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c69b0-e504-4598-8ee8-b6ce886a52d8 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:05.954155 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c69b0-e504-4598-8ee8-b6ce886a52d8 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:05.966053 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6285c-d028-4275-b397-ff4775ed2ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:05.967007 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6285c-d028-4275-b397-ff4775ed2ec1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:05.967300 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6285c-d028-4275-b397-ff4775ed2ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:05.968038 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6285c-d028-4275-b397-ff4775ed2ec1 admin admin] PATH_INFO: `/v3/projects/396c03e6858546ec84114920056c23f3/users/0e4d1ee11bba42228b6d1e0e324bc7fa/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:05.983423 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b6285c-d028-4275-b397-ff4775ed2ec1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=396c03e6858546ec84114920056c23f3, user_id=0e4d1ee11bba42228b6d1e0e324bc7fa, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:05.986108 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b6285c-d028-4275-b397-ff4775ed2ec1 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:06.105695 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1319/2620] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:41:05 2022] PUT /identity/v3/projects/396c03e6858546ec84114920056c23f3/users/0e4d1ee11bba42228b6d1e0e324bc7fa/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:06.121301 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153c81b-6cdd-4838-9322-8e75b3701cb8 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:06.121533 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153c81b-6cdd-4838-9322-8e75b3701cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:06.121829 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153c81b-6cdd-4838-9322-8e75b3701cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:06.149020 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-b153c81b-6cdd-4838-9322-8e75b3701cb8 None None] MFA Rules not processed for user `0e4d1ee11bba42228b6d1e0e324bc7fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:06.150405 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-b153c81b-6cdd-4838-9322-8e75b3701cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:06.218591 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1302/2621] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:06.223303 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6285c-d028-4275-b397-ff4775ed2ec1 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:06.254829 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:06.285533 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6285c-d028-4275-b397-ff4775ed2ec1 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:06.290843 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb4683a-f5e4-43ce-a9dd-a5c0a8f6223d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:06.291488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb4683a-f5e4-43ce-a9dd-a5c0a8f6223d admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:06.291833 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb4683a-f5e4-43ce-a9dd-a5c0a8f6223d admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:06.292117 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb4683a-f5e4-43ce-a9dd-a5c0a8f6223d admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:06.292947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb4683a-f5e4-43ce-a9dd-a5c0a8f6223d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:06.294907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb4683a-f5e4-43ce-a9dd-a5c0a8f6223d admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:06.404417 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1320/2622] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:41:06 2022] POST /identity/v3/projects => generated 379 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:41:06.408664 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b153c81b-6cdd-4838-9322-8e75b3701cb8 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:06.423361 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b153c81b-6cdd-4838-9322-8e75b3701cb8 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:06.478981 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe18f24-cf80-4a3b-9794-4913c154ff4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:06.479676 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe18f24-cf80-4a3b-9794-4913c154ff4d admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:06.479962 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe18f24-cf80-4a3b-9794-4913c154ff4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:06.480239 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe18f24-cf80-4a3b-9794-4913c154ff4d admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:06.480930 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe18f24-cf80-4a3b-9794-4913c154ff4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:06.482831 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe18f24-cf80-4a3b-9794-4913c154ff4d admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:06.492507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:06.597887 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1303/2623] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:41:06 2022] POST /identity/v3/users => generated 290 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:41:06.601956 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb4683a-f5e4-43ce-a9dd-a5c0a8f6223d admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:06.612687 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb4683a-f5e4-43ce-a9dd-a5c0a8f6223d admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:06.617839 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7f7eb-31c9-4bad-9563-7a47b6467b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:06.618498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7f7eb-31c9-4bad-9563-7a47b6467b22 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:06.618782 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7f7eb-31c9-4bad-9563-7a47b6467b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:06.619060 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7f7eb-31c9-4bad-9563-7a47b6467b22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:06.619824 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad7f7eb-31c9-4bad-9563-7a47b6467b22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:06.621448 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad7f7eb-31c9-4bad-9563-7a47b6467b22 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:06.632024 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1321/2624] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:41:06.636797 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe18f24-cf80-4a3b-9794-4913c154ff4d admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:06.646722 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe18f24-cf80-4a3b-9794-4913c154ff4d admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:06.651382 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143aee9f-9c85-45bd-9f1a-4b426fb8aee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:06.652068 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143aee9f-9c85-45bd-9f1a-4b426fb8aee6 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:06.652380 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143aee9f-9c85-45bd-9f1a-4b426fb8aee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:06.652631 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143aee9f-9c85-45bd-9f1a-4b426fb8aee6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:06.653359 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143aee9f-9c85-45bd-9f1a-4b426fb8aee6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:06.654986 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143aee9f-9c85-45bd-9f1a-4b426fb8aee6 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:06.665207 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1304/2625] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:41:06.669270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7f7eb-31c9-4bad-9563-7a47b6467b22 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:06.680211 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7f7eb-31c9-4bad-9563-7a47b6467b22 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:06.684786 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edbc1d7-0a1e-4dfe-94ca-a8e0edc45853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:06.685689 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbc1d7-0a1e-4dfe-94ca-a8e0edc45853 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:06.686149 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbc1d7-0a1e-4dfe-94ca-a8e0edc45853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:06.686426 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbc1d7-0a1e-4dfe-94ca-a8e0edc45853 admin admin] PATH_INFO: `/v3/projects/59177c93b8274ce19cf5950c80754088/users/46ed2fb551dc4b74a9eb8bcdbbbb13d0/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:06.701317 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edbc1d7-0a1e-4dfe-94ca-a8e0edc45853 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59177c93b8274ce19cf5950c80754088, user_id=46ed2fb551dc4b74a9eb8bcdbbbb13d0, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:06.704019 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edbc1d7-0a1e-4dfe-94ca-a8e0edc45853 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:06.791586 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1322/2626] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:41:06 2022] PUT /identity/v3/projects/59177c93b8274ce19cf5950c80754088/users/46ed2fb551dc4b74a9eb8bcdbbbb13d0/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:06.803748 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64256415-3841-417c-8d83-e422c6b75f54 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:06.804621 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64256415-3841-417c-8d83-e422c6b75f54 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:06.804901 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64256415-3841-417c-8d83-e422c6b75f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:06.831756 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-64256415-3841-417c-8d83-e422c6b75f54 None None] MFA Rules not processed for user `46ed2fb551dc4b74a9eb8bcdbbbb13d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:06.832967 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-64256415-3841-417c-8d83-e422c6b75f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:06.896461 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1305/2627] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:41:06.901497 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edbc1d7-0a1e-4dfe-94ca-a8e0edc45853 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:06.963366 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edbc1d7-0a1e-4dfe-94ca-a8e0edc45853 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:06.968040 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7b2f87-96d4-460d-a7fa-eb02e96405a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:06.968701 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b2f87-96d4-460d-a7fa-eb02e96405a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:06.969068 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b2f87-96d4-460d-a7fa-eb02e96405a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:06.971832 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b2f87-96d4-460d-a7fa-eb02e96405a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:06.972624 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7b2f87-96d4-460d-a7fa-eb02e96405a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:06.974607 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7b2f87-96d4-460d-a7fa-eb02e96405a1 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:07.050550 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1323/2628] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:41:06 2022] POST /identity/v3/projects => generated 377 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:41:07.055154 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64256415-3841-417c-8d83-e422c6b75f54 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:07.066559 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64256415-3841-417c-8d83-e422c6b75f54 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:07.129664 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c88ce-f65c-4042-9c6c-dcdf7a2e7989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:07.130458 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c88ce-f65c-4042-9c6c-dcdf7a2e7989 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:07.131066 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c88ce-f65c-4042-9c6c-dcdf7a2e7989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:07.131975 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c88ce-f65c-4042-9c6c-dcdf7a2e7989 admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:07.133510 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c88ce-f65c-4042-9c6c-dcdf7a2e7989 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:07.135917 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c88ce-f65c-4042-9c6c-dcdf7a2e7989 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:07.239095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1306/2629] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:41:07 2022] POST /identity/v3/users => generated 295 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:41:07.243846 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7b2f87-96d4-460d-a7fa-eb02e96405a1 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:07.254345 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7b2f87-96d4-460d-a7fa-eb02e96405a1 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:07.259712 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c062cd4-af42-469e-b367-6fe78d99ef7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:07.260374 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c062cd4-af42-469e-b367-6fe78d99ef7f admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:07.260657 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c062cd4-af42-469e-b367-6fe78d99ef7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:07.260934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c062cd4-af42-469e-b367-6fe78d99ef7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:07.261653 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c062cd4-af42-469e-b367-6fe78d99ef7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:07.263259 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c062cd4-af42-469e-b367-6fe78d99ef7f admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:07.274624 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1324/2630] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:41:07.278965 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c88ce-f65c-4042-9c6c-dcdf7a2e7989 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:07.291058 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c88ce-f65c-4042-9c6c-dcdf7a2e7989 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:07.297455 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd7b77-0471-4d01-a23e-22a612018fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:07.298367 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd7b77-0471-4d01-a23e-22a612018fca admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:07.298651 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd7b77-0471-4d01-a23e-22a612018fca admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:07.300183 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd7b77-0471-4d01-a23e-22a612018fca admin admin] PATH_INFO: `/v3/projects/92ecaba3600145d696296c3cb20d90ca/users/9e67c960032a44658c76b4f6488c70bd/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:07.322789 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd7b77-0471-4d01-a23e-22a612018fca admin admin] RBAC: Authorizing `identity:create_grant(project_id=92ecaba3600145d696296c3cb20d90ca, user_id=9e67c960032a44658c76b4f6488c70bd, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:07.324950 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd7b77-0471-4d01-a23e-22a612018fca admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:07.397058 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1307/2631] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:41:07 2022] PUT /identity/v3/projects/92ecaba3600145d696296c3cb20d90ca/users/9e67c960032a44658c76b4f6488c70bd/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:07.401177 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c062cd4-af42-469e-b367-6fe78d99ef7f admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:07.411193 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c062cd4-af42-469e-b367-6fe78d99ef7f admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:07.459552 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:07.469021 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c56ca-5294-46ff-830c-9d364471daff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:07.469682 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c56ca-5294-46ff-830c-9d364471daff admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:07.469967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c56ca-5294-46ff-830c-9d364471daff admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:07.470245 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c56ca-5294-46ff-830c-9d364471daff admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:07.470966 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c56ca-5294-46ff-830c-9d364471daff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:07.472594 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c56ca-5294-46ff-830c-9d364471daff admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:07.482115 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1325/2632] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:07 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:41:07.486808 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd7b77-0471-4d01-a23e-22a612018fca admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:07.497439 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd7b77-0471-4d01-a23e-22a612018fca admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:07.501972 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afc0c94-aa8d-4fb7-9a2c-789e85281133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:07.502821 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc0c94-aa8d-4fb7-9a2c-789e85281133 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:07.503102 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc0c94-aa8d-4fb7-9a2c-789e85281133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:07.503377 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc0c94-aa8d-4fb7-9a2c-789e85281133 admin admin] PATH_INFO: `/v3/projects/92ecaba3600145d696296c3cb20d90ca/users/9e67c960032a44658c76b4f6488c70bd/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:07.505356 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afc0c94-aa8d-4fb7-9a2c-789e85281133 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92ecaba3600145d696296c3cb20d90ca, user_id=9e67c960032a44658c76b4f6488c70bd, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:07.508567 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afc0c94-aa8d-4fb7-9a2c-789e85281133 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:07.519018 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1308/2633] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:41:07 2022] PUT /identity/v3/projects/92ecaba3600145d696296c3cb20d90ca/users/9e67c960032a44658c76b4f6488c70bd/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:07.583763 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941aa56-7677-496f-a4a8-4f35eb8e198d None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:07.584155 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941aa56-7677-496f-a4a8-4f35eb8e198d None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:07.584444 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941aa56-7677-496f-a4a8-4f35eb8e198d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:07.612921 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-8941aa56-7677-496f-a4a8-4f35eb8e198d None None] MFA Rules not processed for user `9e67c960032a44658c76b4f6488c70bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:07.614202 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-8941aa56-7677-496f-a4a8-4f35eb8e198d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:07.684850 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1326/2634] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:07 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:07.697222 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afc0c94-aa8d-4fb7-9a2c-789e85281133 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:07.699124 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-9afc0c94-aa8d-4fb7-9a2c-789e85281133 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:07.730356 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:07.756816 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afc0c94-aa8d-4fb7-9a2c-789e85281133 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:07.760944 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c15338e-1cc9-4cae-a896-ee8dc8f9b502 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:07.761580 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15338e-1cc9-4cae-a896-ee8dc8f9b502 service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:07.761936 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15338e-1cc9-4cae-a896-ee8dc8f9b502 service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:07.762178 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15338e-1cc9-4cae-a896-ee8dc8f9b502 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:07.771188 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c15338e-1cc9-4cae-a896-ee8dc8f9b502 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:07.773283 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c15338e-1cc9-4cae-a896-ee8dc8f9b502 service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:07.782916 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1309/2635] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:41:07 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:41:07.891759 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8941aa56-7677-496f-a4a8-4f35eb8e198d None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:07.893238 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-8941aa56-7677-496f-a4a8-4f35eb8e198d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:07.905874 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8941aa56-7677-496f-a4a8-4f35eb8e198d None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:07.909349 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6abc9ae-03c0-44a8-b87e-6ec28d1a83d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:07.909975 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6abc9ae-03c0-44a8-b87e-6ec28d1a83d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:07.910254 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6abc9ae-03c0-44a8-b87e-6ec28d1a83d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:07.910528 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6abc9ae-03c0-44a8-b87e-6ec28d1a83d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:07.968365 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6abc9ae-03c0-44a8-b87e-6ec28d1a83d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:07.969815 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6abc9ae-03c0-44a8-b87e-6ec28d1a83d6 service cinder] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:07.979583 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1327/2636] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:41:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:41:08.594743 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c15338e-1cc9-4cae-a896-ee8dc8f9b502 service cinder] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:08.596465 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-5c15338e-1cc9-4cae-a896-ee8dc8f9b502 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:08.671096 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c15338e-1cc9-4cae-a896-ee8dc8f9b502 service cinder] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:08.678559 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c065356-d444-4551-a782-b7239e617c3c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '755c2782cfb042f89950b8fc862e4fa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:08.679205 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c065356-d444-4551-a782-b7239e617c3c service placement] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:08.679478 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c065356-d444-4551-a782-b7239e617c3c service placement] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:08.679834 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c065356-d444-4551-a782-b7239e617c3c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:08.681777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-0c065356-d444-4551-a782-b7239e617c3c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:08.694094 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c065356-d444-4551-a782-b7239e617c3c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:08.696638 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c065356-d444-4551-a782-b7239e617c3c service placement] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:08.706450 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1310/2637] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:41:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:41:10.407994 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6abc9ae-03c0-44a8-b87e-6ec28d1a83d6 service cinder] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:10.410808 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-a6abc9ae-03c0-44a8-b87e-6ec28d1a83d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:10.480608 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6abc9ae-03c0-44a8-b87e-6ec28d1a83d6 service cinder] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:10.485017 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9b4b9c-5802-4d19-bc21-0d538926e197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:10.485881 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b4b9c-5802-4d19-bc21-0d538926e197 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:10.486162 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b4b9c-5802-4d19-bc21-0d538926e197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:10.486766 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b4b9c-5802-4d19-bc21-0d538926e197 admin admin] PATH_INFO: `/v3/users/86a2b7dd44214043ac8096b07fb71b1b` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:10.487977 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9b4b9c-5802-4d19-bc21-0d538926e197 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86a2b7dd44214043ac8096b07fb71b1b)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:10.489877 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9b4b9c-5802-4d19-bc21-0d538926e197 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:10.502870 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:10.779962 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-bb9b4b9c-5802-4d19-bc21-0d538926e197 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86a2b7dd44214043ac8096b07fb71b1b'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:10.784225 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-bb9b4b9c-5802-4d19-bc21-0d538926e197 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86a2b7dd44214043ac8096b07fb71b1b'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:10.787742 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-bb9b4b9c-5802-4d19-bc21-0d538926e197 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86a2b7dd44214043ac8096b07fb71b1b'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:10.788410 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-bb9b4b9c-5802-4d19-bc21-0d538926e197 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86a2b7dd44214043ac8096b07fb71b1b'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:10.801211 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-bb9b4b9c-5802-4d19-bc21-0d538926e197 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86a2b7dd44214043ac8096b07fb71b1b'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:10.889819 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1328/2638] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:41:10 2022] DELETE /identity/v3/users/86a2b7dd44214043ac8096b07fb71b1b => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:41:10.894354 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b6e03-4c77-48cd-952b-3ccd148e2c9c None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:10.894823 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b6e03-4c77-48cd-952b-3ccd148e2c9c None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:10.894924 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b6e03-4c77-48cd-952b-3ccd148e2c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:10.913259 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-679b6e03-4c77-48cd-952b-3ccd148e2c9c None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:10.914544 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-679b6e03-4c77-48cd-952b-3ccd148e2c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:10.983459 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1311/2639] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:10.994102 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9b4b9c-5802-4d19-bc21-0d538926e197 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:10.995655 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-bb9b4b9c-5802-4d19-bc21-0d538926e197 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:11.051130 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9b4b9c-5802-4d19-bc21-0d538926e197 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:11.055440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac225a86-49aa-4c2f-8ac4-dd5f8cdc9db5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:11.056152 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac225a86-49aa-4c2f-8ac4-dd5f8cdc9db5 service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:11.056435 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac225a86-49aa-4c2f-8ac4-dd5f8cdc9db5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:11.061815 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac225a86-49aa-4c2f-8ac4-dd5f8cdc9db5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:11.066068 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac225a86-49aa-4c2f-8ac4-dd5f8cdc9db5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:11.067538 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac225a86-49aa-4c2f-8ac4-dd5f8cdc9db5 service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:11.077307 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1329/2640] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:41:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:41:11.473182 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b6e03-4c77-48cd-952b-3ccd148e2c9c None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:11.477803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-679b6e03-4c77-48cd-952b-3ccd148e2c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:11.490346 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b6e03-4c77-48cd-952b-3ccd148e2c9c None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:11.495645 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6275edd2-0c4a-4d3e-aee0-f8e1c92b721f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:11.496837 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275edd2-0c4a-4d3e-aee0-f8e1c92b721f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:11.497114 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275edd2-0c4a-4d3e-aee0-f8e1c92b721f admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:11.499512 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275edd2-0c4a-4d3e-aee0-f8e1c92b721f admin admin] PATH_INFO: `/v3/projects/3971a67ffb7e435eb069d7ee573d2cb1` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:11.499512 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6275edd2-0c4a-4d3e-aee0-f8e1c92b721f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3971a67ffb7e435eb069d7ee573d2cb1)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:11.500607 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6275edd2-0c4a-4d3e-aee0-f8e1c92b721f admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:11.603119 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-6275edd2-0c4a-4d3e-aee0-f8e1c92b721f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3971a67ffb7e435eb069d7ee573d2cb1', 'request_id': 'req-6275edd2-0c4a-4d3e-aee0-f8e1c92b721f'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:11.607043 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:11.609421 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-6275edd2-0c4a-4d3e-aee0-f8e1c92b721f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3971a67ffb7e435eb069d7ee573d2cb1', 'request_id': 'req-6275edd2-0c4a-4d3e-aee0-f8e1c92b721f'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:11.615916 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-6275edd2-0c4a-4d3e-aee0-f8e1c92b721f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3971a67ffb7e435eb069d7ee573d2cb1', 'request_id': 'req-6275edd2-0c4a-4d3e-aee0-f8e1c92b721f'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:11.665063 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-6275edd2-0c4a-4d3e-aee0-f8e1c92b721f admin admin] The token cache is being invalidate because project 3971a67ffb7e435eb069d7ee573d2cb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:41:11.665346 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-6275edd2-0c4a-4d3e-aee0-f8e1c92b721f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:11.665961 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-6275edd2-0c4a-4d3e-aee0-f8e1c92b721f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:11.667482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1312/2641] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:41:11 2022] DELETE /identity/v3/projects/3971a67ffb7e435eb069d7ee573d2cb1 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:11.671420 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac225a86-49aa-4c2f-8ac4-dd5f8cdc9db5 service neutron] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:11.681041 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac225a86-49aa-4c2f-8ac4-dd5f8cdc9db5 service neutron] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:11.738318 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083baaac-fa22-42be-96f6-64bfeaa18cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:11.739398 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083baaac-fa22-42be-96f6-64bfeaa18cf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:11.740078 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083baaac-fa22-42be-96f6-64bfeaa18cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:11.740354 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083baaac-fa22-42be-96f6-64bfeaa18cf7 admin admin] PATH_INFO: `/v3/users/6159498cb23b4087b2292d650547b043` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:11.741666 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083baaac-fa22-42be-96f6-64bfeaa18cf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6159498cb23b4087b2292d650547b043)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:11.743605 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083baaac-fa22-42be-96f6-64bfeaa18cf7 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:11.917080 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:11.925695 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-083baaac-fa22-42be-96f6-64bfeaa18cf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6159498cb23b4087b2292d650547b043'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:11.926543 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-083baaac-fa22-42be-96f6-64bfeaa18cf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6159498cb23b4087b2292d650547b043'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:11.933790 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-083baaac-fa22-42be-96f6-64bfeaa18cf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6159498cb23b4087b2292d650547b043'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:11.934494 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-083baaac-fa22-42be-96f6-64bfeaa18cf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6159498cb23b4087b2292d650547b043'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:11.946606 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-083baaac-fa22-42be-96f6-64bfeaa18cf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6159498cb23b4087b2292d650547b043'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:12.058260 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6275edd2-0c4a-4d3e-aee0-f8e1c92b721f admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:12.060021 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-6275edd2-0c4a-4d3e-aee0-f8e1c92b721f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:12.067700 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1330/2642] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:41:11 2022] DELETE /identity/v3/users/6159498cb23b4087b2292d650547b043 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:12.073817 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6275edd2-0c4a-4d3e-aee0-f8e1c92b721f admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:12.141915 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd37bd-c5ff-44d4-a391-646260a1dcc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:12.142887 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd37bd-c5ff-44d4-a391-646260a1dcc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:12.143170 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd37bd-c5ff-44d4-a391-646260a1dcc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:12.143599 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd37bd-c5ff-44d4-a391-646260a1dcc7 admin admin] PATH_INFO: `/v3/users/0e4d1ee11bba42228b6d1e0e324bc7fa` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:12.144845 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd37bd-c5ff-44d4-a391-646260a1dcc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e4d1ee11bba42228b6d1e0e324bc7fa)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:12.146880 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd37bd-c5ff-44d4-a391-646260a1dcc7 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:12.323157 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-a6dd37bd-c5ff-44d4-a391-646260a1dcc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e4d1ee11bba42228b6d1e0e324bc7fa'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:12.330896 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-a6dd37bd-c5ff-44d4-a391-646260a1dcc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e4d1ee11bba42228b6d1e0e324bc7fa'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:12.343244 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-a6dd37bd-c5ff-44d4-a391-646260a1dcc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e4d1ee11bba42228b6d1e0e324bc7fa'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:12.344048 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-a6dd37bd-c5ff-44d4-a391-646260a1dcc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e4d1ee11bba42228b6d1e0e324bc7fa'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:12.449203 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-a6dd37bd-c5ff-44d4-a391-646260a1dcc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e4d1ee11bba42228b6d1e0e324bc7fa'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:12.455860 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1313/2643] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:41:12 2022] DELETE /identity/v3/users/0e4d1ee11bba42228b6d1e0e324bc7fa => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:12.460568 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc834f-3c46-4f7d-b9a1-3fc79586787b None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:12.460857 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc834f-3c46-4f7d-b9a1-3fc79586787b None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:12.461131 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc834f-3c46-4f7d-b9a1-3fc79586787b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:12.478432 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-10fc834f-3c46-4f7d-b9a1-3fc79586787b None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:12.479661 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-10fc834f-3c46-4f7d-b9a1-3fc79586787b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:12.538424 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1331/2644] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:12.547674 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd37bd-c5ff-44d4-a391-646260a1dcc7 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:12.549082 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-a6dd37bd-c5ff-44d4-a391-646260a1dcc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:12.564612 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc834f-3c46-4f7d-b9a1-3fc79586787b None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:12.566047 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-10fc834f-3c46-4f7d-b9a1-3fc79586787b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:12.581939 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc834f-3c46-4f7d-b9a1-3fc79586787b None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:12.588196 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d152af71-d95b-4373-907e-62710d97a637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:12.589073 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152af71-d95b-4373-907e-62710d97a637 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:12.589353 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152af71-d95b-4373-907e-62710d97a637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:12.589623 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152af71-d95b-4373-907e-62710d97a637 admin admin] PATH_INFO: `/v3/projects/9d0c8a212f7f4a8f8b49ae71eb5419ae` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:12.591385 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d152af71-d95b-4373-907e-62710d97a637 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d0c8a212f7f4a8f8b49ae71eb5419ae)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:12.593577 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d152af71-d95b-4373-907e-62710d97a637 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:12.619342 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd37bd-c5ff-44d4-a391-646260a1dcc7 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:12.623361 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f58993-8566-4da5-86b1-3bb2c44f1add service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:12.624069 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f58993-8566-4da5-86b1-3bb2c44f1add service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:12.624376 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f58993-8566-4da5-86b1-3bb2c44f1add service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:12.624648 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f58993-8566-4da5-86b1-3bb2c44f1add service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:12.634478 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f58993-8566-4da5-86b1-3bb2c44f1add service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:12.635991 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f58993-8566-4da5-86b1-3bb2c44f1add service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:12.647994 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1314/2645] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:41:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:41:12.735550 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-d152af71-d95b-4373-907e-62710d97a637 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d0c8a212f7f4a8f8b49ae71eb5419ae', 'request_id': 'req-d152af71-d95b-4373-907e-62710d97a637'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:12.741775 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-d152af71-d95b-4373-907e-62710d97a637 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d0c8a212f7f4a8f8b49ae71eb5419ae', 'request_id': 'req-d152af71-d95b-4373-907e-62710d97a637'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:12.747996 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-d152af71-d95b-4373-907e-62710d97a637 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d0c8a212f7f4a8f8b49ae71eb5419ae', 'request_id': 'req-d152af71-d95b-4373-907e-62710d97a637'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:12.865883 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-d152af71-d95b-4373-907e-62710d97a637 admin admin] The token cache is being invalidate because project 9d0c8a212f7f4a8f8b49ae71eb5419ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:41:12.866266 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-d152af71-d95b-4373-907e-62710d97a637 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:12.867135 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-d152af71-d95b-4373-907e-62710d97a637 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:12.869005 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1332/2646] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:41:12 2022] DELETE /identity/v3/projects/9d0c8a212f7f4a8f8b49ae71eb5419ae => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:12.873733 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e1475-97b5-44b9-81eb-484eb24eae77 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:12.874096 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e1475-97b5-44b9-81eb-484eb24eae77 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:12.874451 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e1475-97b5-44b9-81eb-484eb24eae77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:12.882029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:12.894403 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-153e1475-97b5-44b9-81eb-484eb24eae77 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:12.895672 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-153e1475-97b5-44b9-81eb-484eb24eae77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:12.968406 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1315/2647] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:12.975964 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474088b-7f72-46c8-9fe4-159c3abf0420 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:12.976255 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474088b-7f72-46c8-9fe4-159c3abf0420 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:12.977501 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474088b-7f72-46c8-9fe4-159c3abf0420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:12.994759 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-8474088b-7f72-46c8-9fe4-159c3abf0420 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:12.996048 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-8474088b-7f72-46c8-9fe4-159c3abf0420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:13.004623 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1333/2648] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:13.009554 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e1475-97b5-44b9-81eb-484eb24eae77 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:13.019184 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e1475-97b5-44b9-81eb-484eb24eae77 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:13.024412 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b45d11a-c2b4-4163-9cd4-e0dbc3df4ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:13.025052 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45d11a-c2b4-4163-9cd4-e0dbc3df4ba5 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:13.025334 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45d11a-c2b4-4163-9cd4-e0dbc3df4ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:13.025608 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45d11a-c2b4-4163-9cd4-e0dbc3df4ba5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:13.026333 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b45d11a-c2b4-4163-9cd4-e0dbc3df4ba5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:41:13.026640 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b45d11a-c2b4-4163-9cd4-e0dbc3df4ba5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:13.028285 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b45d11a-c2b4-4163-9cd4-e0dbc3df4ba5 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:13.040416 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1316/2649] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:41:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:41:13.044527 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8474088b-7f72-46c8-9fe4-159c3abf0420 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:13.053561 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8474088b-7f72-46c8-9fe4-159c3abf0420 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:13.058189 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ad921-5a47-4433-8096-6a60d36f74d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:13.062450 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ad921-5a47-4433-8096-6a60d36f74d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:13.062450 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ad921-5a47-4433-8096-6a60d36f74d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:13.062450 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ad921-5a47-4433-8096-6a60d36f74d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:13.062450 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0ad921-5a47-4433-8096-6a60d36f74d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:13.062450 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0ad921-5a47-4433-8096-6a60d36f74d1 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:13.085687 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b45d11a-c2b4-4163-9cd4-e0dbc3df4ba5 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:13.087489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-5b45d11a-c2b4-4163-9cd4-e0dbc3df4ba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:13.101697 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b45d11a-c2b4-4163-9cd4-e0dbc3df4ba5 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:13.106488 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef907a-10df-4707-871d-cee1e513b90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:13.107375 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef907a-10df-4707-871d-cee1e513b90c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:13.107699 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef907a-10df-4707-871d-cee1e513b90c admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:13.107977 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef907a-10df-4707-871d-cee1e513b90c admin admin] PATH_INFO: `/v3/projects/396c03e6858546ec84114920056c23f3` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:13.109199 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ef907a-10df-4707-871d-cee1e513b90c admin admin] RBAC: Authorizing `identity:delete_project(project_id=396c03e6858546ec84114920056c23f3)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:13.111113 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ef907a-10df-4707-871d-cee1e513b90c admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:13.170324 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1334/2650] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:41:13 2022] POST /identity/v3/projects => generated 409 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:41:13.174246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ad921-5a47-4433-8096-6a60d36f74d1 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:13.183464 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:13.231416 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-f5ef907a-10df-4707-871d-cee1e513b90c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '396c03e6858546ec84114920056c23f3', 'request_id': 'req-f5ef907a-10df-4707-871d-cee1e513b90c'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:13.236728 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ad921-5a47-4433-8096-6a60d36f74d1 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:13.238544 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-f5ef907a-10df-4707-871d-cee1e513b90c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '396c03e6858546ec84114920056c23f3', 'request_id': 'req-f5ef907a-10df-4707-871d-cee1e513b90c'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:13.246108 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-f5ef907a-10df-4707-871d-cee1e513b90c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '396c03e6858546ec84114920056c23f3', 'request_id': 'req-f5ef907a-10df-4707-871d-cee1e513b90c'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:13.292258 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a022b0e-54b8-44b7-973b-fdf37a27eb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:13.292954 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a022b0e-54b8-44b7-973b-fdf37a27eb52 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:13.293296 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a022b0e-54b8-44b7-973b-fdf37a27eb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:13.293575 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a022b0e-54b8-44b7-973b-fdf37a27eb52 admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:13.294262 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a022b0e-54b8-44b7-973b-fdf37a27eb52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:13.296228 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a022b0e-54b8-44b7-973b-fdf37a27eb52 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:13.323010 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-f5ef907a-10df-4707-871d-cee1e513b90c admin admin] The token cache is being invalidate because project 396c03e6858546ec84114920056c23f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:41:13.323299 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-f5ef907a-10df-4707-871d-cee1e513b90c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:13.323977 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-f5ef907a-10df-4707-871d-cee1e513b90c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:13.325607 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1317/2651] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:41:13 2022] DELETE /identity/v3/projects/396c03e6858546ec84114920056c23f3 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:41:13.329448 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef907a-10df-4707-871d-cee1e513b90c admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:13.330749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-f5ef907a-10df-4707-871d-cee1e513b90c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:13.348122 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef907a-10df-4707-871d-cee1e513b90c admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:13.353404 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ead5f-82e2-4d57-a970-4829b8b2ac69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:13.356340 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ead5f-82e2-4d57-a970-4829b8b2ac69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:13.356340 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ead5f-82e2-4d57-a970-4829b8b2ac69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:13.356340 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ead5f-82e2-4d57-a970-4829b8b2ac69 admin admin] PATH_INFO: `/v3/users/46ed2fb551dc4b74a9eb8bcdbbbb13d0` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:13.356340 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7ead5f-82e2-4d57-a970-4829b8b2ac69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46ed2fb551dc4b74a9eb8bcdbbbb13d0)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:13.357968 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7ead5f-82e2-4d57-a970-4829b8b2ac69 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:13.409263 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1335/2652] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:41:13 2022] POST /identity/v3/users => generated 305 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:41:13.413863 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a022b0e-54b8-44b7-973b-fdf37a27eb52 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:13.415440 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-8a022b0e-54b8-44b7-973b-fdf37a27eb52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:13.427393 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a022b0e-54b8-44b7-973b-fdf37a27eb52 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:13.431726 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db906583-3974-4512-9136-01f669d79e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:13.432367 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db906583-3974-4512-9136-01f669d79e9b admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:13.432649 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db906583-3974-4512-9136-01f669d79e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:13.432924 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db906583-3974-4512-9136-01f669d79e9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:13.433640 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db906583-3974-4512-9136-01f669d79e9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:13.435218 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db906583-3974-4512-9136-01f669d79e9b admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:13.444979 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1336/2653] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:41:13.448881 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db906583-3974-4512-9136-01f669d79e9b admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:13.458078 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db906583-3974-4512-9136-01f669d79e9b admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:13.463290 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045664d-fcd7-4971-b890-77662ea8d4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:13.463924 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045664d-fcd7-4971-b890-77662ea8d4a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:13.464205 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045664d-fcd7-4971-b890-77662ea8d4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:13.464475 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045664d-fcd7-4971-b890-77662ea8d4a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:13.465215 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045664d-fcd7-4971-b890-77662ea8d4a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:13.466745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045664d-fcd7-4971-b890-77662ea8d4a5 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:13.477690 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1337/2654] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:41:13.481703 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045664d-fcd7-4971-b890-77662ea8d4a5 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:13.492415 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045664d-fcd7-4971-b890-77662ea8d4a5 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:13.497009 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec07bf04-5272-43c2-aff4-3f3907449d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:13.497833 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07bf04-5272-43c2-aff4-3f3907449d73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:13.498169 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07bf04-5272-43c2-aff4-3f3907449d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:13.498451 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07bf04-5272-43c2-aff4-3f3907449d73 admin admin] PATH_INFO: `/v3/projects/87026759d5cb40d99dadff7f37572891/users/52014a3ad5694287bca6f62b9b17987a/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:13.513035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec07bf04-5272-43c2-aff4-3f3907449d73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87026759d5cb40d99dadff7f37572891, user_id=52014a3ad5694287bca6f62b9b17987a, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:13.515700 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec07bf04-5272-43c2-aff4-3f3907449d73 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:13.566225 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-4c7ead5f-82e2-4d57-a970-4829b8b2ac69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46ed2fb551dc4b74a9eb8bcdbbbb13d0'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:13.573746 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-4c7ead5f-82e2-4d57-a970-4829b8b2ac69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46ed2fb551dc4b74a9eb8bcdbbbb13d0'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:13.584723 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:13.588207 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-4c7ead5f-82e2-4d57-a970-4829b8b2ac69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46ed2fb551dc4b74a9eb8bcdbbbb13d0'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:13.588335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1338/2655] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:41:13 2022] PUT /identity/v3/projects/87026759d5cb40d99dadff7f37572891/users/52014a3ad5694287bca6f62b9b17987a/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:41:13.588441 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-4c7ead5f-82e2-4d57-a970-4829b8b2ac69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46ed2fb551dc4b74a9eb8bcdbbbb13d0'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:13.602615 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdab872-39c0-4c33-a6d1-3a6e2e89d701 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:13.602909 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdab872-39c0-4c33-a6d1-3a6e2e89d701 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:13.603188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdab872-39c0-4c33-a6d1-3a6e2e89d701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:13.629976 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-cbdab872-39c0-4c33-a6d1-3a6e2e89d701 None None] MFA Rules not processed for user `52014a3ad5694287bca6f62b9b17987a`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:13.631234 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-cbdab872-39c0-4c33-a6d1-3a6e2e89d701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:13.676012 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:13.695709 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1339/2656] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:13 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:41:13.699968 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdab872-39c0-4c33-a6d1-3a6e2e89d701 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:13.700783 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-4c7ead5f-82e2-4d57-a970-4829b8b2ac69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46ed2fb551dc4b74a9eb8bcdbbbb13d0'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:13.707214 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1318/2657] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:41:13 2022] DELETE /identity/v3/users/46ed2fb551dc4b74a9eb8bcdbbbb13d0 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:41:13.762006 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdab872-39c0-4c33-a6d1-3a6e2e89d701 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:13.821235 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efef0c0-0070-429b-8bff-6c32490f919f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:13.821899 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efef0c0-0070-429b-8bff-6c32490f919f admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:13.822181 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efef0c0-0070-429b-8bff-6c32490f919f admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:13.822455 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efef0c0-0070-429b-8bff-6c32490f919f admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:13.823171 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efef0c0-0070-429b-8bff-6c32490f919f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:13.825110 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efef0c0-0070-429b-8bff-6c32490f919f admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:13.948136 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1340/2658] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:41:13 2022] POST /identity/v3/projects => generated 409 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:41:13.949311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ead5f-82e2-4d57-a970-4829b8b2ac69 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:13.955407 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-4c7ead5f-82e2-4d57-a970-4829b8b2ac69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:13.974787 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ead5f-82e2-4d57-a970-4829b8b2ac69 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:14.029013 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a98c0-fb38-41ba-8eb9-7aa7fd4f496f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:14.029687 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a98c0-fb38-41ba-8eb9-7aa7fd4f496f admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:14.029965 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a98c0-fb38-41ba-8eb9-7aa7fd4f496f admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:14.030298 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a98c0-fb38-41ba-8eb9-7aa7fd4f496f admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:14.030980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3a98c0-fb38-41ba-8eb9-7aa7fd4f496f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:14.032921 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3a98c0-fb38-41ba-8eb9-7aa7fd4f496f admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:14.129248 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1319/2659] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:41:13 2022] POST /identity/v3/users => generated 311 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:41:14.133383 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efef0c0-0070-429b-8bff-6c32490f919f admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:14.143855 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efef0c0-0070-429b-8bff-6c32490f919f admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:14.149035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1b4ac-d5bf-4dec-8520-c5d238c0fb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:14.149745 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1b4ac-d5bf-4dec-8520-c5d238c0fb21 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:14.150023 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1b4ac-d5bf-4dec-8520-c5d238c0fb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:14.150292 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1b4ac-d5bf-4dec-8520-c5d238c0fb21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:14.151019 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee1b4ac-d5bf-4dec-8520-c5d238c0fb21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:14.152740 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee1b4ac-d5bf-4dec-8520-c5d238c0fb21 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:14.165686 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a98c0-fb38-41ba-8eb9-7aa7fd4f496f admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:14.165961 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1341/2660] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:41:14.168078 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-4b3a98c0-fb38-41ba-8eb9-7aa7fd4f496f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:14.169685 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1b4ac-d5bf-4dec-8520-c5d238c0fb21 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:14.181645 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a98c0-fb38-41ba-8eb9-7aa7fd4f496f admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:14.185368 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1b4ac-d5bf-4dec-8520-c5d238c0fb21 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:14.186929 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a45d20-2987-49ab-a092-fbcafbb59555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:14.187906 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a45d20-2987-49ab-a092-fbcafbb59555 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:14.188190 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a45d20-2987-49ab-a092-fbcafbb59555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:14.188513 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a45d20-2987-49ab-a092-fbcafbb59555 admin admin] PATH_INFO: `/v3/projects/59177c93b8274ce19cf5950c80754088` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:14.189843 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a45d20-2987-49ab-a092-fbcafbb59555 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59177c93b8274ce19cf5950c80754088)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:14.190988 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087cad7-b6b2-4495-a119-0c4ae52ef14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:14.191905 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087cad7-b6b2-4495-a119-0c4ae52ef14f admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:14.192189 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087cad7-b6b2-4495-a119-0c4ae52ef14f admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:14.192463 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087cad7-b6b2-4495-a119-0c4ae52ef14f admin admin] PATH_INFO: `/v3/projects/391de79963154e85a1f05702945ca782/users/1f89c6c045954e55aaabc46c21ff4767/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:14.192590 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a45d20-2987-49ab-a092-fbcafbb59555 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:14.208477 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1087cad7-b6b2-4495-a119-0c4ae52ef14f admin admin] RBAC: Authorizing `identity:create_grant(project_id=391de79963154e85a1f05702945ca782, user_id=1f89c6c045954e55aaabc46c21ff4767, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:14.211808 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1087cad7-b6b2-4495-a119-0c4ae52ef14f admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:14.310829 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1342/2661] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:41:14 2022] PUT /identity/v3/projects/391de79963154e85a1f05702945ca782/users/1f89c6c045954e55aaabc46c21ff4767/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:41:14.314631 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087cad7-b6b2-4495-a119-0c4ae52ef14f admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:14.336783 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-87a45d20-2987-49ab-a092-fbcafbb59555 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59177c93b8274ce19cf5950c80754088', 'request_id': 'req-87a45d20-2987-49ab-a092-fbcafbb59555'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:14.342704 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-87a45d20-2987-49ab-a092-fbcafbb59555 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59177c93b8274ce19cf5950c80754088', 'request_id': 'req-87a45d20-2987-49ab-a092-fbcafbb59555'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:14.349544 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-87a45d20-2987-49ab-a092-fbcafbb59555 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59177c93b8274ce19cf5950c80754088', 'request_id': 'req-87a45d20-2987-49ab-a092-fbcafbb59555'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:14.361395 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:14.378119 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087cad7-b6b2-4495-a119-0c4ae52ef14f admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:14.415250 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-87a45d20-2987-49ab-a092-fbcafbb59555 admin admin] The token cache is being invalidate because project 59177c93b8274ce19cf5950c80754088 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:41:14.415540 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-87a45d20-2987-49ab-a092-fbcafbb59555 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:14.416213 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-87a45d20-2987-49ab-a092-fbcafbb59555 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:14.417778 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1320/2662] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:41:14 2022] DELETE /identity/v3/projects/59177c93b8274ce19cf5950c80754088 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:41:14.421690 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a45d20-2987-49ab-a092-fbcafbb59555 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:14.423029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-87a45d20-2987-49ab-a092-fbcafbb59555 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:14.435924 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67551e-e120-4361-926c-63cbe6331c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:14.436602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67551e-e120-4361-926c-63cbe6331c4d admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:14.436892 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67551e-e120-4361-926c-63cbe6331c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:14.437172 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67551e-e120-4361-926c-63cbe6331c4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:14.437905 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df67551e-e120-4361-926c-63cbe6331c4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:14.439516 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df67551e-e120-4361-926c-63cbe6331c4d admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:14.452287 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1343/2663] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:14 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:41:14.456288 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67551e-e120-4361-926c-63cbe6331c4d admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:14.457982 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-df67551e-e120-4361-926c-63cbe6331c4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:14.470761 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67551e-e120-4361-926c-63cbe6331c4d admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:14.475435 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf06ae7-7121-4172-b487-6a8c852a2f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:14.476852 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf06ae7-7121-4172-b487-6a8c852a2f54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:14.477137 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf06ae7-7121-4172-b487-6a8c852a2f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:14.477417 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf06ae7-7121-4172-b487-6a8c852a2f54 admin admin] PATH_INFO: `/v3/projects/391de79963154e85a1f05702945ca782/users/1f89c6c045954e55aaabc46c21ff4767/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:14.479697 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf06ae7-7121-4172-b487-6a8c852a2f54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=391de79963154e85a1f05702945ca782, user_id=1f89c6c045954e55aaabc46c21ff4767, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:14.482054 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf06ae7-7121-4172-b487-6a8c852a2f54 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:14.489210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a45d20-2987-49ab-a092-fbcafbb59555 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:14.493470 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:DELETE=9,keystone:SELECT=86,keystone:INSERT=5 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:14.493600 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1344/2664] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:41:14 2022] PUT /identity/v3/projects/391de79963154e85a1f05702945ca782/users/1f89c6c045954e55aaabc46c21ff4767/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:41:14.551356 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd5135-a93d-467e-948c-ba85bb4dccad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:14.552276 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd5135-a93d-467e-948c-ba85bb4dccad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:14.552560 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd5135-a93d-467e-948c-ba85bb4dccad admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:14.552869 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd5135-a93d-467e-948c-ba85bb4dccad admin admin] PATH_INFO: `/v3/users/9e67c960032a44658c76b4f6488c70bd` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:14.554812 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fd5135-a93d-467e-948c-ba85bb4dccad admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e67c960032a44658c76b4f6488c70bd)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:14.556932 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fd5135-a93d-467e-948c-ba85bb4dccad admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:14.560134 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab52c57-334b-4341-a1d0-5f05492f1316 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:14.560423 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab52c57-334b-4341-a1d0-5f05492f1316 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:14.560697 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab52c57-334b-4341-a1d0-5f05492f1316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:14.586613 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-7ab52c57-334b-4341-a1d0-5f05492f1316 None None] MFA Rules not processed for user `1f89c6c045954e55aaabc46c21ff4767`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:14.588666 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-7ab52c57-334b-4341-a1d0-5f05492f1316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:14.655972 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1345/2665] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:14 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:41:14.666813 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab52c57-334b-4341-a1d0-5f05492f1316 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:14.668403 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-7ab52c57-334b-4341-a1d0-5f05492f1316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:14.725410 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab52c57-334b-4341-a1d0-5f05492f1316 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:14.728990 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd99df16-7b5a-4e58-a969-97c785396663 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:14.729629 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99df16-7b5a-4e58-a969-97c785396663 service cinder] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:14.729914 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99df16-7b5a-4e58-a969-97c785396663 service cinder] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:14.730195 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99df16-7b5a-4e58-a969-97c785396663 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:14.739868 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd99df16-7b5a-4e58-a969-97c785396663 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:14.741635 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd99df16-7b5a-4e58-a969-97c785396663 service cinder] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:14.748179 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-76fd5135-a93d-467e-948c-ba85bb4dccad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e67c960032a44658c76b4f6488c70bd'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:14.751261 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1346/2666] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:41:14 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 15:41:14.757542 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-76fd5135-a93d-467e-948c-ba85bb4dccad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e67c960032a44658c76b4f6488c70bd'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:14.770762 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-76fd5135-a93d-467e-948c-ba85bb4dccad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e67c960032a44658c76b4f6488c70bd'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:14.771490 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-76fd5135-a93d-467e-948c-ba85bb4dccad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e67c960032a44658c76b4f6488c70bd'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:14.870876 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-76fd5135-a93d-467e-948c-ba85bb4dccad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e67c960032a44658c76b4f6488c70bd'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:14.877404 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1321/2667] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:41:14 2022] DELETE /identity/v3/users/9e67c960032a44658c76b4f6488c70bd => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:41:15.090432 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd99df16-7b5a-4e58-a969-97c785396663 service cinder] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:15.094564 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:15.100077 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd99df16-7b5a-4e58-a969-97c785396663 service cinder] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:15.151176 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d3d1d-7a1f-4b2c-994a-688ce6ac8ff9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:15.151972 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d3d1d-7a1f-4b2c-994a-688ce6ac8ff9 service cinder] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:15.152328 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d3d1d-7a1f-4b2c-994a-688ce6ac8ff9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:15.152677 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d3d1d-7a1f-4b2c-994a-688ce6ac8ff9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:15.154493 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-b64d3d1d-7a1f-4b2c-994a-688ce6ac8ff9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:15.212573 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64d3d1d-7a1f-4b2c-994a-688ce6ac8ff9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:15.214065 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64d3d1d-7a1f-4b2c-994a-688ce6ac8ff9 service cinder] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:15.223671 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1347/2668] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:41:15 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 15:41:15.280042 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd5135-a93d-467e-948c-ba85bb4dccad admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:15.282077 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-76fd5135-a93d-467e-948c-ba85bb4dccad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:15.363979 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd5135-a93d-467e-948c-ba85bb4dccad admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:15.370690 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad6f6f0-a9f6-470e-85b1-5608bb9b4b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:15.371553 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad6f6f0-a9f6-470e-85b1-5608bb9b4b16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:15.371864 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad6f6f0-a9f6-470e-85b1-5608bb9b4b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:15.373195 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad6f6f0-a9f6-470e-85b1-5608bb9b4b16 admin admin] PATH_INFO: `/v3/projects/92ecaba3600145d696296c3cb20d90ca` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:15.374860 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad6f6f0-a9f6-470e-85b1-5608bb9b4b16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92ecaba3600145d696296c3cb20d90ca)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:15.378168 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad6f6f0-a9f6-470e-85b1-5608bb9b4b16 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:15.480232 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:15.496644 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-cad6f6f0-a9f6-470e-85b1-5608bb9b4b16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92ecaba3600145d696296c3cb20d90ca', 'request_id': 'req-cad6f6f0-a9f6-470e-85b1-5608bb9b4b16'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:15.501960 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-cad6f6f0-a9f6-470e-85b1-5608bb9b4b16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92ecaba3600145d696296c3cb20d90ca', 'request_id': 'req-cad6f6f0-a9f6-470e-85b1-5608bb9b4b16'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:15.508659 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-cad6f6f0-a9f6-470e-85b1-5608bb9b4b16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92ecaba3600145d696296c3cb20d90ca', 'request_id': 'req-cad6f6f0-a9f6-470e-85b1-5608bb9b4b16'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:15.585731 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-cad6f6f0-a9f6-470e-85b1-5608bb9b4b16 admin admin] The token cache is being invalidate because project 92ecaba3600145d696296c3cb20d90ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:41:15.586021 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-cad6f6f0-a9f6-470e-85b1-5608bb9b4b16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:15.586628 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-cad6f6f0-a9f6-470e-85b1-5608bb9b4b16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:15.588207 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1322/2669] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:41:15 2022] DELETE /identity/v3/projects/92ecaba3600145d696296c3cb20d90ca => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:15.594411 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdb20f-90d8-4604-93fd-5f91880d31e0 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:15.594701 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdb20f-90d8-4604-93fd-5f91880d31e0 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:15.594983 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdb20f-90d8-4604-93fd-5f91880d31e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:15.612902 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-8acdb20f-90d8-4604-93fd-5f91880d31e0 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:15.614137 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-8acdb20f-90d8-4604-93fd-5f91880d31e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:15.681445 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1348/2670] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:41:15.690563 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee25dff-340e-4d18-95bb-92680f84e9a0 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:15.690859 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee25dff-340e-4d18-95bb-92680f84e9a0 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:15.691139 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee25dff-340e-4d18-95bb-92680f84e9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:15.711827 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-5ee25dff-340e-4d18-95bb-92680f84e9a0 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:15.713046 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-5ee25dff-340e-4d18-95bb-92680f84e9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:15.721459 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1323/2671] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:15.744561 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acdb20f-90d8-4604-93fd-5f91880d31e0 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:15.744561 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acdb20f-90d8-4604-93fd-5f91880d31e0 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:15.748045 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf769eb-c935-4860-a174-49b9f6d1d1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:15.748610 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf769eb-c935-4860-a174-49b9f6d1d1c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:15.748892 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf769eb-c935-4860-a174-49b9f6d1d1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:15.749203 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf769eb-c935-4860-a174-49b9f6d1d1c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:15.750091 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf769eb-c935-4860-a174-49b9f6d1d1c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:41:15.750403 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf769eb-c935-4860-a174-49b9f6d1d1c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:15.752072 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf769eb-c935-4860-a174-49b9f6d1d1c9 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:15.764403 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1349/2672] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:41:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:41:15.768900 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee25dff-340e-4d18-95bb-92680f84e9a0 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:15.778137 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee25dff-340e-4d18-95bb-92680f84e9a0 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:15.782758 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66578b7-e796-4968-ae8d-47ddecc58079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:15.783398 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66578b7-e796-4968-ae8d-47ddecc58079 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:15.783724 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66578b7-e796-4968-ae8d-47ddecc58079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:15.784000 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66578b7-e796-4968-ae8d-47ddecc58079 admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:15.784706 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66578b7-e796-4968-ae8d-47ddecc58079 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:15.786576 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66578b7-e796-4968-ae8d-47ddecc58079 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:15.984411 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1324/2673] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:41:15 2022] POST /identity/v3/projects => generated 369 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:41:15.987413 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf769eb-c935-4860-a174-49b9f6d1d1c9 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:15.997453 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf769eb-c935-4860-a174-49b9f6d1d1c9 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:16.039176 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:16.053534 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d7c3a-29e8-4483-b71f-8051e1f45264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:16.054159 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d7c3a-29e8-4483-b71f-8051e1f45264 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:16.054442 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d7c3a-29e8-4483-b71f-8051e1f45264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:16.054718 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d7c3a-29e8-4483-b71f-8051e1f45264 admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:16.055408 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d7c3a-29e8-4483-b71f-8051e1f45264 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:16.057351 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d7c3a-29e8-4483-b71f-8051e1f45264 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:16.146728 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1350/2674] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:41:15 2022] POST /identity/v3/users => generated 285 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:41:16.151406 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66578b7-e796-4968-ae8d-47ddecc58079 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:16.161816 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66578b7-e796-4968-ae8d-47ddecc58079 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:16.166814 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e33bd-4698-4b00-be26-08092a9a56b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:16.167481 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e33bd-4698-4b00-be26-08092a9a56b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:16.167797 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e33bd-4698-4b00-be26-08092a9a56b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:16.168075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e33bd-4698-4b00-be26-08092a9a56b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:16.169100 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e33bd-4698-4b00-be26-08092a9a56b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:16.170455 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e33bd-4698-4b00-be26-08092a9a56b9 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:16.181550 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1325/2675] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:41:16.185643 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d7c3a-29e8-4483-b71f-8051e1f45264 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:16.196495 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d7c3a-29e8-4483-b71f-8051e1f45264 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:16.201264 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc42d5c-b1cb-41d0-a325-383615c1f2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:16.201935 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc42d5c-b1cb-41d0-a325-383615c1f2a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:16.202277 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc42d5c-b1cb-41d0-a325-383615c1f2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:16.202558 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc42d5c-b1cb-41d0-a325-383615c1f2a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:16.203292 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc42d5c-b1cb-41d0-a325-383615c1f2a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:16.204932 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc42d5c-b1cb-41d0-a325-383615c1f2a6 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:16.217612 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1351/2676] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:41:16.221930 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e33bd-4698-4b00-be26-08092a9a56b9 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:16.232619 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e33bd-4698-4b00-be26-08092a9a56b9 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:16.237579 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60bf0d9-37d3-4fac-ad5f-3c190ac3c552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:16.238461 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60bf0d9-37d3-4fac-ad5f-3c190ac3c552 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:16.238738 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60bf0d9-37d3-4fac-ad5f-3c190ac3c552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:16.239009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60bf0d9-37d3-4fac-ad5f-3c190ac3c552 admin admin] PATH_INFO: `/v3/projects/e01375189eb04e17bacc1c3892b86b9e/users/a7e54f97003c4818a87d6c49bd83614d/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:16.254464 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60bf0d9-37d3-4fac-ad5f-3c190ac3c552 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e01375189eb04e17bacc1c3892b86b9e, user_id=a7e54f97003c4818a87d6c49bd83614d, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:16.257166 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60bf0d9-37d3-4fac-ad5f-3c190ac3c552 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:16.354295 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1326/2677] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:41:16 2022] PUT /identity/v3/projects/e01375189eb04e17bacc1c3892b86b9e/users/a7e54f97003c4818a87d6c49bd83614d/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:16.359517 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51058161-07d2-4199-96a7-3186b0aedeb3 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:16.359823 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51058161-07d2-4199-96a7-3186b0aedeb3 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:16.360092 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51058161-07d2-4199-96a7-3186b0aedeb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:16.377347 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-51058161-07d2-4199-96a7-3186b0aedeb3 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:16.378592 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-51058161-07d2-4199-96a7-3186b0aedeb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:16.437589 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1352/2678] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:16.447324 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60bf0d9-37d3-4fac-ad5f-3c190ac3c552 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:16.449112 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-c60bf0d9-37d3-4fac-ad5f-3c190ac3c552 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:16.506683 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60bf0d9-37d3-4fac-ad5f-3c190ac3c552 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:16.510516 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607177a3-6f2f-4932-92cc-0e92d862564c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:16.511107 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607177a3-6f2f-4932-92cc-0e92d862564c service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:16.511393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607177a3-6f2f-4932-92cc-0e92d862564c service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:16.511709 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607177a3-6f2f-4932-92cc-0e92d862564c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:16.520833 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607177a3-6f2f-4932-92cc-0e92d862564c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:16.522784 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607177a3-6f2f-4932-92cc-0e92d862564c service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:16.532393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1327/2679] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:41:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:41:17.696326 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015125cc-44a6-4e4d-90ec-57c9fb847d87 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:17.698147 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015125cc-44a6-4e4d-90ec-57c9fb847d87 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:17.698147 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015125cc-44a6-4e4d-90ec-57c9fb847d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:17.723523 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-015125cc-44a6-4e4d-90ec-57c9fb847d87 None None] MFA Rules not processed for user `a7e54f97003c4818a87d6c49bd83614d`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:17.726894 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-015125cc-44a6-4e4d-90ec-57c9fb847d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:17.790541 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1353/2680] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:17 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:17.795100 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607177a3-6f2f-4932-92cc-0e92d862564c service neutron] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:17.805308 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607177a3-6f2f-4932-92cc-0e92d862564c service neutron] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:17.809866 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb7a37-fb7e-411c-98af-dd2f0118723a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:17.810504 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb7a37-fb7e-411c-98af-dd2f0118723a admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:17.810787 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb7a37-fb7e-411c-98af-dd2f0118723a admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:17.811063 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb7a37-fb7e-411c-98af-dd2f0118723a admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:17.811769 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cb7a37-fb7e-411c-98af-dd2f0118723a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:17.813656 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cb7a37-fb7e-411c-98af-dd2f0118723a admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:17.942073 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1328/2681] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:41:17 2022] POST /identity/v3/projects => generated 371 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:41:17.946779 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015125cc-44a6-4e4d-90ec-57c9fb847d87 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:17.957607 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015125cc-44a6-4e4d-90ec-57c9fb847d87 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:18.004671 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:18.014019 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de6f9ea-c393-416e-8ac8-b829626e51bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:18.024153 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6f9ea-c393-416e-8ac8-b829626e51bc admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:18.024153 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6f9ea-c393-416e-8ac8-b829626e51bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:18.024153 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6f9ea-c393-416e-8ac8-b829626e51bc admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:18.024153 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de6f9ea-c393-416e-8ac8-b829626e51bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:18.024153 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de6f9ea-c393-416e-8ac8-b829626e51bc admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:18.134978 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1354/2682] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:41:17 2022] POST /identity/v3/users => generated 292 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:41:18.139047 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb7a37-fb7e-411c-98af-dd2f0118723a admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:18.149883 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb7a37-fb7e-411c-98af-dd2f0118723a admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:18.154891 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f2c9f-7410-413a-a9be-9536e11e86e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:18.155534 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f2c9f-7410-413a-a9be-9536e11e86e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:18.155848 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f2c9f-7410-413a-a9be-9536e11e86e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:18.156120 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f2c9f-7410-413a-a9be-9536e11e86e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:18.156874 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f2c9f-7410-413a-a9be-9536e11e86e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:18.158451 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f2c9f-7410-413a-a9be-9536e11e86e1 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:18.170304 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1329/2683] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:41:18.174395 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de6f9ea-c393-416e-8ac8-b829626e51bc admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:18.184579 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de6f9ea-c393-416e-8ac8-b829626e51bc admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:18.189237 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61c060-6aa9-4b9c-9ebd-74d2bc103a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:18.190103 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61c060-6aa9-4b9c-9ebd-74d2bc103a21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:18.190383 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61c060-6aa9-4b9c-9ebd-74d2bc103a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:18.190661 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61c060-6aa9-4b9c-9ebd-74d2bc103a21 admin admin] PATH_INFO: `/v3/projects/2818f02432344270b2ae3c1212432e01/users/6f549d6a5831404ea646a4129ee1d4ab/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:18.206249 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f61c060-6aa9-4b9c-9ebd-74d2bc103a21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2818f02432344270b2ae3c1212432e01, user_id=6f549d6a5831404ea646a4129ee1d4ab, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:18.208965 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f61c060-6aa9-4b9c-9ebd-74d2bc103a21 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:18.288221 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1355/2684] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:41:18 2022] PUT /identity/v3/projects/2818f02432344270b2ae3c1212432e01/users/6f549d6a5831404ea646a4129ee1d4ab/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:18.292135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f2c9f-7410-413a-a9be-9536e11e86e1 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:18.302275 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f2c9f-7410-413a-a9be-9536e11e86e1 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:18.321095 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:18.358749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f024ee-2d13-4373-aa07-8cc47f1e207c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:18.359407 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f024ee-2d13-4373-aa07-8cc47f1e207c admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:18.359725 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f024ee-2d13-4373-aa07-8cc47f1e207c admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:18.360005 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f024ee-2d13-4373-aa07-8cc47f1e207c admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:18.360728 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f024ee-2d13-4373-aa07-8cc47f1e207c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:18.362619 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f024ee-2d13-4373-aa07-8cc47f1e207c admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:18.372957 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1330/2685] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:18 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:41:18.377041 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61c060-6aa9-4b9c-9ebd-74d2bc103a21 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:18.388240 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61c060-6aa9-4b9c-9ebd-74d2bc103a21 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:18.392899 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8ae23-6014-4d4e-8333-107f302f75b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:18.393772 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8ae23-6014-4d4e-8333-107f302f75b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:18.394056 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8ae23-6014-4d4e-8333-107f302f75b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:18.394328 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8ae23-6014-4d4e-8333-107f302f75b8 admin admin] PATH_INFO: `/v3/projects/2818f02432344270b2ae3c1212432e01/users/6f549d6a5831404ea646a4129ee1d4ab/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:18.396346 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb8ae23-6014-4d4e-8333-107f302f75b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2818f02432344270b2ae3c1212432e01, user_id=6f549d6a5831404ea646a4129ee1d4ab, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:18.399620 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb8ae23-6014-4d4e-8333-107f302f75b8 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:18.409952 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1356/2686] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:41:18 2022] PUT /identity/v3/projects/2818f02432344270b2ae3c1212432e01/users/6f549d6a5831404ea646a4129ee1d4ab/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:19.716435 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4246b-2b13-40cf-bd72-951577d8a026 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:19.716970 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4246b-2b13-40cf-bd72-951577d8a026 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:19.717409 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4246b-2b13-40cf-bd72-951577d8a026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:19.749608 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-9bc4246b-2b13-40cf-bd72-951577d8a026 None None] MFA Rules not processed for user `6f549d6a5831404ea646a4129ee1d4ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:19.751037 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-9bc4246b-2b13-40cf-bd72-951577d8a026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:19.840826 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1331/2687] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:19 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:19.853481 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8ae23-6014-4d4e-8333-107f302f75b8 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:19.855263 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-3bb8ae23-6014-4d4e-8333-107f302f75b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:19.916509 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8ae23-6014-4d4e-8333-107f302f75b8 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:19.920147 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce07780-73bb-4070-aaaf-85a26349c650 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:19.920791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce07780-73bb-4070-aaaf-85a26349c650 service cinder] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:19.921124 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce07780-73bb-4070-aaaf-85a26349c650 service cinder] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:19.921399 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce07780-73bb-4070-aaaf-85a26349c650 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:19.979675 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce07780-73bb-4070-aaaf-85a26349c650 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:19.981121 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce07780-73bb-4070-aaaf-85a26349c650 service cinder] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:19.990304 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1357/2688] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:41:19 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 15:41:23.827400 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4246b-2b13-40cf-bd72-951577d8a026 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:23.828846 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-9bc4246b-2b13-40cf-bd72-951577d8a026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:23.841281 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4246b-2b13-40cf-bd72-951577d8a026 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:23.844812 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cded315-c1bb-4bd9-bf26-8f7c26040031 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:23.845440 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cded315-c1bb-4bd9-bf26-8f7c26040031 service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:23.845729 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cded315-c1bb-4bd9-bf26-8f7c26040031 service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:23.846008 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cded315-c1bb-4bd9-bf26-8f7c26040031 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:23.855556 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cded315-c1bb-4bd9-bf26-8f7c26040031 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:23.856988 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cded315-c1bb-4bd9-bf26-8f7c26040031 service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:23.867003 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1332/2689] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:41:23 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:41:26.516410 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce07780-73bb-4070-aaaf-85a26349c650 service cinder] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:26.519802 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-1ce07780-73bb-4070-aaaf-85a26349c650 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:26.588243 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce07780-73bb-4070-aaaf-85a26349c650 service cinder] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:26.592736 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde446d5-d3c1-4bd8-835c-a4f0cd7131e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:26.593612 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde446d5-d3c1-4bd8-835c-a4f0cd7131e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:26.593876 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde446d5-d3c1-4bd8-835c-a4f0cd7131e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:26.594147 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde446d5-d3c1-4bd8-835c-a4f0cd7131e6 admin admin] PATH_INFO: `/v3/users/52014a3ad5694287bca6f62b9b17987a` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:26.595304 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde446d5-d3c1-4bd8-835c-a4f0cd7131e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52014a3ad5694287bca6f62b9b17987a)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:26.597237 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde446d5-d3c1-4bd8-835c-a4f0cd7131e6 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:26.598695 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:26.794179 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-fde446d5-d3c1-4bd8-835c-a4f0cd7131e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52014a3ad5694287bca6f62b9b17987a'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:26.794853 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-fde446d5-d3c1-4bd8-835c-a4f0cd7131e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52014a3ad5694287bca6f62b9b17987a'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:26.801917 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-fde446d5-d3c1-4bd8-835c-a4f0cd7131e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52014a3ad5694287bca6f62b9b17987a'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:26.802544 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-fde446d5-d3c1-4bd8-835c-a4f0cd7131e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52014a3ad5694287bca6f62b9b17987a'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:26.815040 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-fde446d5-d3c1-4bd8-835c-a4f0cd7131e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52014a3ad5694287bca6f62b9b17987a'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:26.923527 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1358/2690] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:41:26 2022] DELETE /identity/v3/users/52014a3ad5694287bca6f62b9b17987a => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:26.928192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbc886-f71f-44d1-9541-18fe3d5a9024 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:26.928693 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbc886-f71f-44d1-9541-18fe3d5a9024 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:26.928782 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbc886-f71f-44d1-9541-18fe3d5a9024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:26.953547 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-a7bbc886-f71f-44d1-9541-18fe3d5a9024 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:26.954923 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-a7bbc886-f71f-44d1-9541-18fe3d5a9024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:27.016634 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1333/2691] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:27.027847 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde446d5-d3c1-4bd8-835c-a4f0cd7131e6 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:27.029258 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-fde446d5-d3c1-4bd8-835c-a4f0cd7131e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:27.099676 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde446d5-d3c1-4bd8-835c-a4f0cd7131e6 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:27.104558 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3763b5af-ab04-4c0f-9093-e7725a334f31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:27.105301 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763b5af-ab04-4c0f-9093-e7725a334f31 service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:27.105649 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763b5af-ab04-4c0f-9093-e7725a334f31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:27.105990 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763b5af-ab04-4c0f-9093-e7725a334f31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:27.115753 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3763b5af-ab04-4c0f-9093-e7725a334f31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:27.117295 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3763b5af-ab04-4c0f-9093-e7725a334f31 service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:27.126943 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1359/2692] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:41:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:41:27.553342 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bbc886-f71f-44d1-9541-18fe3d5a9024 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:27.555331 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-a7bbc886-f71f-44d1-9541-18fe3d5a9024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:27.571385 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bbc886-f71f-44d1-9541-18fe3d5a9024 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:27.577581 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d8af8-e33b-4416-b26b-b01143719fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:27.579431 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d8af8-e33b-4416-b26b-b01143719fc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:27.579755 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d8af8-e33b-4416-b26b-b01143719fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:27.580032 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d8af8-e33b-4416-b26b-b01143719fc0 admin admin] PATH_INFO: `/v3/projects/87026759d5cb40d99dadff7f37572891` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:27.581229 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36d8af8-e33b-4416-b26b-b01143719fc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87026759d5cb40d99dadff7f37572891)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:27.583133 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36d8af8-e33b-4416-b26b-b01143719fc0 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:27.667959 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:27.684706 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-f36d8af8-e33b-4416-b26b-b01143719fc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87026759d5cb40d99dadff7f37572891', 'request_id': 'req-f36d8af8-e33b-4416-b26b-b01143719fc0'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:27.690108 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-f36d8af8-e33b-4416-b26b-b01143719fc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87026759d5cb40d99dadff7f37572891', 'request_id': 'req-f36d8af8-e33b-4416-b26b-b01143719fc0'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:27.697079 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-f36d8af8-e33b-4416-b26b-b01143719fc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87026759d5cb40d99dadff7f37572891', 'request_id': 'req-f36d8af8-e33b-4416-b26b-b01143719fc0'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:27.784169 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-f36d8af8-e33b-4416-b26b-b01143719fc0 admin admin] The token cache is being invalidate because project 87026759d5cb40d99dadff7f37572891 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:41:27.784463 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-f36d8af8-e33b-4416-b26b-b01143719fc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:27.785070 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-f36d8af8-e33b-4416-b26b-b01143719fc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:27.786646 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1334/2693] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:41:27 2022] DELETE /identity/v3/projects/87026759d5cb40d99dadff7f37572891 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:27.790820 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3763b5af-ab04-4c0f-9093-e7725a334f31 service neutron] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:27.800881 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3763b5af-ab04-4c0f-9093-e7725a334f31 service neutron] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:27.857680 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6431b511-bfba-4050-8c80-c5bc9f078e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:27.858601 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431b511-bfba-4050-8c80-c5bc9f078e87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:27.858883 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431b511-bfba-4050-8c80-c5bc9f078e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:27.859150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431b511-bfba-4050-8c80-c5bc9f078e87 admin admin] PATH_INFO: `/v3/users/1f89c6c045954e55aaabc46c21ff4767` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:27.860345 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6431b511-bfba-4050-8c80-c5bc9f078e87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f89c6c045954e55aaabc46c21ff4767)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:27.862562 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6431b511-bfba-4050-8c80-c5bc9f078e87 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:27.886682 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:28.136139 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-6431b511-bfba-4050-8c80-c5bc9f078e87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f89c6c045954e55aaabc46c21ff4767'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:28.136718 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-6431b511-bfba-4050-8c80-c5bc9f078e87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f89c6c045954e55aaabc46c21ff4767'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:28.143501 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-6431b511-bfba-4050-8c80-c5bc9f078e87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f89c6c045954e55aaabc46c21ff4767'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:28.144147 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-6431b511-bfba-4050-8c80-c5bc9f078e87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f89c6c045954e55aaabc46c21ff4767'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:28.155819 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-6431b511-bfba-4050-8c80-c5bc9f078e87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f89c6c045954e55aaabc46c21ff4767'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:28.232703 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1360/2694] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:41:27 2022] DELETE /identity/v3/users/1f89c6c045954e55aaabc46c21ff4767 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:28.633827 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d8af8-e33b-4416-b26b-b01143719fc0 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:28.638353 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-f36d8af8-e33b-4416-b26b-b01143719fc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:28.650152 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d8af8-e33b-4416-b26b-b01143719fc0 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:28.714929 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c88910-a3fe-48f7-b84a-75a79bd8b730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:28.716049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c88910-a3fe-48f7-b84a-75a79bd8b730 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:28.716790 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c88910-a3fe-48f7-b84a-75a79bd8b730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:28.717076 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c88910-a3fe-48f7-b84a-75a79bd8b730 admin admin] PATH_INFO: `/v3/projects/391de79963154e85a1f05702945ca782` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:28.718434 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c88910-a3fe-48f7-b84a-75a79bd8b730 admin admin] RBAC: Authorizing `identity:delete_project(project_id=391de79963154e85a1f05702945ca782)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:28.720622 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c88910-a3fe-48f7-b84a-75a79bd8b730 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:28.871253 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-83c88910-a3fe-48f7-b84a-75a79bd8b730 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '391de79963154e85a1f05702945ca782', 'request_id': 'req-83c88910-a3fe-48f7-b84a-75a79bd8b730'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:28.876700 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-83c88910-a3fe-48f7-b84a-75a79bd8b730 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '391de79963154e85a1f05702945ca782', 'request_id': 'req-83c88910-a3fe-48f7-b84a-75a79bd8b730'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:28.883374 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-83c88910-a3fe-48f7-b84a-75a79bd8b730 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '391de79963154e85a1f05702945ca782', 'request_id': 'req-83c88910-a3fe-48f7-b84a-75a79bd8b730'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:29.015046 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-83c88910-a3fe-48f7-b84a-75a79bd8b730 admin admin] The token cache is being invalidate because project 391de79963154e85a1f05702945ca782 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:41:29.015559 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-83c88910-a3fe-48f7-b84a-75a79bd8b730 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:29.016075 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-83c88910-a3fe-48f7-b84a-75a79bd8b730 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:29.017867 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1335/2695] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:41:28 2022] DELETE /identity/v3/projects/391de79963154e85a1f05702945ca782 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:29.023650 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3463514d-c3fd-4f3f-8a45-fda2c2f75f8d None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:29.024160 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3463514d-c3fd-4f3f-8a45-fda2c2f75f8d None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:29.024246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3463514d-c3fd-4f3f-8a45-fda2c2f75f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:29.046574 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-3463514d-c3fd-4f3f-8a45-fda2c2f75f8d None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:29.047986 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-3463514d-c3fd-4f3f-8a45-fda2c2f75f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:29.111368 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1361/2696] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:29.119698 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b273b3e-08a0-4b53-8758-c06281edc218 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:29.119989 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b273b3e-08a0-4b53-8758-c06281edc218 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:29.120266 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b273b3e-08a0-4b53-8758-c06281edc218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:29.138656 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-6b273b3e-08a0-4b53-8758-c06281edc218 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:29.139892 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-6b273b3e-08a0-4b53-8758-c06281edc218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:29.147709 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1336/2697] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:29.151757 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3463514d-c3fd-4f3f-8a45-fda2c2f75f8d None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:29.163008 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3463514d-c3fd-4f3f-8a45-fda2c2f75f8d None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:29.166967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd585d3-af1d-41aa-a70b-9ddb275e054a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:29.167653 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd585d3-af1d-41aa-a70b-9ddb275e054a admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:29.167954 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd585d3-af1d-41aa-a70b-9ddb275e054a admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:29.168206 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd585d3-af1d-41aa-a70b-9ddb275e054a admin admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:29.168925 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd585d3-af1d-41aa-a70b-9ddb275e054a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:41:29.169233 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd585d3-af1d-41aa-a70b-9ddb275e054a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:29.170813 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd585d3-af1d-41aa-a70b-9ddb275e054a admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:29.183196 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1362/2698] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:41:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:41:29.187389 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b273b3e-08a0-4b53-8758-c06281edc218 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:29.198915 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b273b3e-08a0-4b53-8758-c06281edc218 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:29.202784 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af16538-bd20-4fa1-b334-b68d3c6cc6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:29.203434 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af16538-bd20-4fa1-b334-b68d3c6cc6bf admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:29.203759 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af16538-bd20-4fa1-b334-b68d3c6cc6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:29.204040 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af16538-bd20-4fa1-b334-b68d3c6cc6bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:29.205039 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af16538-bd20-4fa1-b334-b68d3c6cc6bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:29.206916 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af16538-bd20-4fa1-b334-b68d3c6cc6bf admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:29.274062 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1337/2699] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:41:29 2022] POST /identity/v3/projects => generated 385 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:41:29.276430 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd585d3-af1d-41aa-a70b-9ddb275e054a admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:29.287378 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd585d3-af1d-41aa-a70b-9ddb275e054a admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:29.346288 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8599e5e-8f66-4bad-b6bd-1c76ff0db6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:29.346956 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8599e5e-8f66-4bad-b6bd-1c76ff0db6c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:29.347240 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8599e5e-8f66-4bad-b6bd-1c76ff0db6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:29.347510 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8599e5e-8f66-4bad-b6bd-1c76ff0db6c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:29.348641 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8599e5e-8f66-4bad-b6bd-1c76ff0db6c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:29.350685 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8599e5e-8f66-4bad-b6bd-1c76ff0db6c9 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:29.426152 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1363/2700] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:41:29 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:41:29.430792 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af16538-bd20-4fa1-b334-b68d3c6cc6bf admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:29.441313 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af16538-bd20-4fa1-b334-b68d3c6cc6bf admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:29.446125 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b71489-769e-49af-bce0-da33a7281eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:29.446784 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b71489-769e-49af-bce0-da33a7281eea admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:29.447066 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b71489-769e-49af-bce0-da33a7281eea admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:29.447342 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b71489-769e-49af-bce0-da33a7281eea admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:29.448118 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b71489-769e-49af-bce0-da33a7281eea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:29.449812 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b71489-769e-49af-bce0-da33a7281eea admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:29.463548 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1338/2701] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:41:29.464858 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8599e5e-8f66-4bad-b6bd-1c76ff0db6c9 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:29.475247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8599e5e-8f66-4bad-b6bd-1c76ff0db6c9 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:29.480496 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6eda31-25d6-4fcd-854f-f90ecd99dbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:29.481154 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6eda31-25d6-4fcd-854f-f90ecd99dbd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:29.481437 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6eda31-25d6-4fcd-854f-f90ecd99dbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:29.481741 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6eda31-25d6-4fcd-854f-f90ecd99dbd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:29.482649 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6eda31-25d6-4fcd-854f-f90ecd99dbd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:29.484304 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6eda31-25d6-4fcd-854f-f90ecd99dbd0 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:29.495326 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1364/2702] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:41:29.499380 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b71489-769e-49af-bce0-da33a7281eea admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:29.506304 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:29.510059 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b71489-769e-49af-bce0-da33a7281eea admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:29.514678 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf56b56-b605-45b7-b479-c0b053552ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:29.519771 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf56b56-b605-45b7-b479-c0b053552ae1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:29.519771 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf56b56-b605-45b7-b479-c0b053552ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:29.519771 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf56b56-b605-45b7-b479-c0b053552ae1 admin admin] PATH_INFO: `/v3/projects/d42b0f77aabb4d3da882df3e96e60772/users/8fabbe11263f4a918922471d8c9675c2/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:29.530650 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf56b56-b605-45b7-b479-c0b053552ae1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d42b0f77aabb4d3da882df3e96e60772, user_id=8fabbe11263f4a918922471d8c9675c2, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:29.533404 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf56b56-b605-45b7-b479-c0b053552ae1 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:29.608275 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1339/2703] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:41:29 2022] PUT /identity/v3/projects/d42b0f77aabb4d3da882df3e96e60772/users/8fabbe11263f4a918922471d8c9675c2/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:29.619024 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc38f5d-69e7-4581-a85c-9e64cdbe31e1 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:29.619317 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc38f5d-69e7-4581-a85c-9e64cdbe31e1 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:29.619627 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc38f5d-69e7-4581-a85c-9e64cdbe31e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:29.632655 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:29.646573 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-dfc38f5d-69e7-4581-a85c-9e64cdbe31e1 None None] MFA Rules not processed for user `8fabbe11263f4a918922471d8c9675c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:29.648492 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-dfc38f5d-69e7-4581-a85c-9e64cdbe31e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:29.712367 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1365/2704] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:29.723871 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf56b56-b605-45b7-b479-c0b053552ae1 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:29.725665 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-0bf56b56-b605-45b7-b479-c0b053552ae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:29.783463 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf56b56-b605-45b7-b479-c0b053552ae1 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:29.787108 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110f7e3-7ca7-4e8c-a9e8-65c1a621e282 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:29.787767 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110f7e3-7ca7-4e8c-a9e8-65c1a621e282 service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:29.788063 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110f7e3-7ca7-4e8c-a9e8-65c1a621e282 service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:29.788341 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110f7e3-7ca7-4e8c-a9e8-65c1a621e282 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:29.797043 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8110f7e3-7ca7-4e8c-a9e8-65c1a621e282 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:29.798637 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8110f7e3-7ca7-4e8c-a9e8-65c1a621e282 service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:29.808089 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1340/2705] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:41:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:41:29.827619 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc38f5d-69e7-4581-a85c-9e64cdbe31e1 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:29.888980 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc38f5d-69e7-4581-a85c-9e64cdbe31e1 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:29.893582 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b519919-97a7-4350-9b53-de9fa2b5967c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:29.894449 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b519919-97a7-4350-9b53-de9fa2b5967c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:29.894720 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b519919-97a7-4350-9b53-de9fa2b5967c admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:29.895043 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b519919-97a7-4350-9b53-de9fa2b5967c admin admin] PATH_INFO: `/v3/users/8fabbe11263f4a918922471d8c9675c2` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:29.896242 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b519919-97a7-4350-9b53-de9fa2b5967c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fabbe11263f4a918922471d8c9675c2)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:29.898196 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b519919-97a7-4350-9b53-de9fa2b5967c admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:30.067722 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-7b519919-97a7-4350-9b53-de9fa2b5967c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fabbe11263f4a918922471d8c9675c2'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:30.068395 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-7b519919-97a7-4350-9b53-de9fa2b5967c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fabbe11263f4a918922471d8c9675c2'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:30.075222 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-7b519919-97a7-4350-9b53-de9fa2b5967c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fabbe11263f4a918922471d8c9675c2'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:30.076079 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-7b519919-97a7-4350-9b53-de9fa2b5967c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fabbe11263f4a918922471d8c9675c2'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:30.088493 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-7b519919-97a7-4350-9b53-de9fa2b5967c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fabbe11263f4a918922471d8c9675c2'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:30.151442 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1366/2706] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:41:29 2022] DELETE /identity/v3/users/8fabbe11263f4a918922471d8c9675c2 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:30.155998 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe96462-14ab-4d65-adbf-ab6916da3be4 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:30.156289 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe96462-14ab-4d65-adbf-ab6916da3be4 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:30.156567 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe96462-14ab-4d65-adbf-ab6916da3be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:30.174266 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-bbe96462-14ab-4d65-adbf-ab6916da3be4 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:30.175466 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-bbe96462-14ab-4d65-adbf-ab6916da3be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:30.234105 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1341/2707] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:30.243847 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b519919-97a7-4350-9b53-de9fa2b5967c admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:30.245252 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-7b519919-97a7-4350-9b53-de9fa2b5967c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:30.264836 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:30.303304 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b519919-97a7-4350-9b53-de9fa2b5967c admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:30.307579 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e469c513-ddc2-4421-9ab0-91487b80fdd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:30.308232 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e469c513-ddc2-4421-9ab0-91487b80fdd0 service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:30.308516 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e469c513-ddc2-4421-9ab0-91487b80fdd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:30.308792 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e469c513-ddc2-4421-9ab0-91487b80fdd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:30.317671 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e469c513-ddc2-4421-9ab0-91487b80fdd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:30.319843 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e469c513-ddc2-4421-9ab0-91487b80fdd0 service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:30.329561 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1367/2708] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:41:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:41:30.744562 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe96462-14ab-4d65-adbf-ab6916da3be4 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:30.744562 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-bbe96462-14ab-4d65-adbf-ab6916da3be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:30.757226 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe96462-14ab-4d65-adbf-ab6916da3be4 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:30.761732 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccb1ebd-7c4e-4ce0-aefb-e17280d142fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:30.762773 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb1ebd-7c4e-4ce0-aefb-e17280d142fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:30.763243 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb1ebd-7c4e-4ce0-aefb-e17280d142fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:30.763512 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb1ebd-7c4e-4ce0-aefb-e17280d142fb admin admin] PATH_INFO: `/v3/projects/d42b0f77aabb4d3da882df3e96e60772` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:30.765309 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccb1ebd-7c4e-4ce0-aefb-e17280d142fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d42b0f77aabb4d3da882df3e96e60772)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:30.767702 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccb1ebd-7c4e-4ce0-aefb-e17280d142fb admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:30.878572 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-3ccb1ebd-7c4e-4ce0-aefb-e17280d142fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd42b0f77aabb4d3da882df3e96e60772', 'request_id': 'req-3ccb1ebd-7c4e-4ce0-aefb-e17280d142fb'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:30.884085 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-3ccb1ebd-7c4e-4ce0-aefb-e17280d142fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd42b0f77aabb4d3da882df3e96e60772', 'request_id': 'req-3ccb1ebd-7c4e-4ce0-aefb-e17280d142fb'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:30.890193 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-3ccb1ebd-7c4e-4ce0-aefb-e17280d142fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd42b0f77aabb4d3da882df3e96e60772', 'request_id': 'req-3ccb1ebd-7c4e-4ce0-aefb-e17280d142fb'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:31.007881 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-3ccb1ebd-7c4e-4ce0-aefb-e17280d142fb admin admin] The token cache is being invalidate because project d42b0f77aabb4d3da882df3e96e60772 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:41:31.007881 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-3ccb1ebd-7c4e-4ce0-aefb-e17280d142fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:31.007881 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-3ccb1ebd-7c4e-4ce0-aefb-e17280d142fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:31.007881 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1342/2709] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:41:30 2022] DELETE /identity/v3/projects/d42b0f77aabb4d3da882df3e96e60772 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:31.010357 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff3613-8455-4a0c-9496-8145597bb451 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:31.010643 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff3613-8455-4a0c-9496-8145597bb451 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:31.010917 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff3613-8455-4a0c-9496-8145597bb451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:31.028299 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-b9ff3613-8455-4a0c-9496-8145597bb451 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:31.029525 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-b9ff3613-8455-4a0c-9496-8145597bb451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:31.088327 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1368/2710] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:31.096877 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e763c-b682-4242-bf53-d3baeab04ea7 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:31.097163 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e763c-b682-4242-bf53-d3baeab04ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:31.097438 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e763c-b682-4242-bf53-d3baeab04ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:31.112032 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:31.117472 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-c91e763c-b682-4242-bf53-d3baeab04ea7 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:31.118657 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-c91e763c-b682-4242-bf53-d3baeab04ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:31.126402 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1343/2711] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:31.130498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff3613-8455-4a0c-9496-8145597bb451 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:31.140810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff3613-8455-4a0c-9496-8145597bb451 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:31.145371 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b765daa-45c3-40b9-8ef1-58ae7b94747d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:31.146018 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b765daa-45c3-40b9-8ef1-58ae7b94747d admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:31.146297 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b765daa-45c3-40b9-8ef1-58ae7b94747d admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:31.154860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b765daa-45c3-40b9-8ef1-58ae7b94747d admin admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:31.154860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b765daa-45c3-40b9-8ef1-58ae7b94747d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:41:31.154860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b765daa-45c3-40b9-8ef1-58ae7b94747d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:31.154860 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b765daa-45c3-40b9-8ef1-58ae7b94747d admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:31.161220 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1369/2712] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:41:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:41:31.165392 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91e763c-b682-4242-bf53-d3baeab04ea7 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:31.174299 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91e763c-b682-4242-bf53-d3baeab04ea7 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:31.179587 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1824018-c4dc-4801-a3b9-3437635b9df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:31.180250 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1824018-c4dc-4801-a3b9-3437635b9df7 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:31.180535 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1824018-c4dc-4801-a3b9-3437635b9df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:31.180812 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1824018-c4dc-4801-a3b9-3437635b9df7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:31.181507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1824018-c4dc-4801-a3b9-3437635b9df7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:31.183361 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1824018-c4dc-4801-a3b9-3437635b9df7 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:31.261819 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1344/2713] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:41:31 2022] POST /identity/v3/projects => generated 375 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:41:31.265713 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b765daa-45c3-40b9-8ef1-58ae7b94747d admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:31.276326 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b765daa-45c3-40b9-8ef1-58ae7b94747d admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:31.330949 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48641566-f97a-4cd3-b334-4ea933f63d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:31.331639 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48641566-f97a-4cd3-b334-4ea933f63d05 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:31.331925 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48641566-f97a-4cd3-b334-4ea933f63d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:31.332203 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48641566-f97a-4cd3-b334-4ea933f63d05 admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:31.332898 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48641566-f97a-4cd3-b334-4ea933f63d05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:31.334809 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48641566-f97a-4cd3-b334-4ea933f63d05 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:31.409048 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1370/2714] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:41:31 2022] POST /identity/v3/users => generated 288 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:41:31.413129 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1824018-c4dc-4801-a3b9-3437635b9df7 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:31.423664 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1824018-c4dc-4801-a3b9-3437635b9df7 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:31.428328 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a2be3-3d34-402c-994f-f56f2eb6a623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:31.429486 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a2be3-3d34-402c-994f-f56f2eb6a623 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:31.429764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a2be3-3d34-402c-994f-f56f2eb6a623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:31.430034 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a2be3-3d34-402c-994f-f56f2eb6a623 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:31.430764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a2be3-3d34-402c-994f-f56f2eb6a623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:31.432374 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a2be3-3d34-402c-994f-f56f2eb6a623 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:31.443062 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1345/2715] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:41:31.448047 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48641566-f97a-4cd3-b334-4ea933f63d05 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:31.457915 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48641566-f97a-4cd3-b334-4ea933f63d05 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:31.462702 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b7bdb-3233-42a8-a6ce-52121ed87492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:31.466443 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b7bdb-3233-42a8-a6ce-52121ed87492 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:31.466443 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b7bdb-3233-42a8-a6ce-52121ed87492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:31.466443 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b7bdb-3233-42a8-a6ce-52121ed87492 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:31.466443 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5b7bdb-3233-42a8-a6ce-52121ed87492 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:31.466443 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5b7bdb-3233-42a8-a6ce-52121ed87492 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:31.476402 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1371/2716] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:41:31.480320 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a2be3-3d34-402c-994f-f56f2eb6a623 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:31.490388 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a2be3-3d34-402c-994f-f56f2eb6a623 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:31.495485 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0908dd-1278-4d0c-ba37-567ca88b6769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:31.496382 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0908dd-1278-4d0c-ba37-567ca88b6769 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:31.496660 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0908dd-1278-4d0c-ba37-567ca88b6769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:31.496930 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0908dd-1278-4d0c-ba37-567ca88b6769 admin admin] PATH_INFO: `/v3/projects/cc35b009de744e7298b272f82377e8a8/users/e4657f81cba64ff9a0385f46b75bd40a/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:31.512226 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0908dd-1278-4d0c-ba37-567ca88b6769 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc35b009de744e7298b272f82377e8a8, user_id=e4657f81cba64ff9a0385f46b75bd40a, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:31.514798 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0908dd-1278-4d0c-ba37-567ca88b6769 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:31.601974 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1346/2717] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:41:31 2022] PUT /identity/v3/projects/cc35b009de744e7298b272f82377e8a8/users/e4657f81cba64ff9a0385f46b75bd40a/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:31.612477 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e28ea-99e5-4852-a49f-4fcd2affac53 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:31.612770 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e28ea-99e5-4852-a49f-4fcd2affac53 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:31.613049 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e28ea-99e5-4852-a49f-4fcd2affac53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:31.640104 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-183e28ea-99e5-4852-a49f-4fcd2affac53 None None] MFA Rules not processed for user `e4657f81cba64ff9a0385f46b75bd40a`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:31.641307 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-183e28ea-99e5-4852-a49f-4fcd2affac53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:31.652756 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:31.705757 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1372/2718] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:31.710253 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0908dd-1278-4d0c-ba37-567ca88b6769 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:31.778342 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0908dd-1278-4d0c-ba37-567ca88b6769 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:31.783053 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c00f59-7df4-403a-a97b-daa3dccc7d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:31.783745 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c00f59-7df4-403a-a97b-daa3dccc7d1b admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:31.784028 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c00f59-7df4-403a-a97b-daa3dccc7d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:31.784300 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c00f59-7df4-403a-a97b-daa3dccc7d1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:31.784982 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c00f59-7df4-403a-a97b-daa3dccc7d1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:31.786905 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c00f59-7df4-403a-a97b-daa3dccc7d1b admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:31.890130 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1347/2719] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:41:31 2022] POST /identity/v3/projects => generated 373 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:41:31.895712 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e28ea-99e5-4852-a49f-4fcd2affac53 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:31.906921 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e28ea-99e5-4852-a49f-4fcd2affac53 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:31.963519 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08df045-7216-4ca2-9675-402684fca537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:31.964218 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08df045-7216-4ca2-9675-402684fca537 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:31.964500 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08df045-7216-4ca2-9675-402684fca537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:31.964776 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08df045-7216-4ca2-9675-402684fca537 admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:31.965497 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08df045-7216-4ca2-9675-402684fca537 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:31.967405 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08df045-7216-4ca2-9675-402684fca537 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:32.060863 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1373/2720] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:41:31 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:41:32.064789 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c00f59-7df4-403a-a97b-daa3dccc7d1b admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:32.075242 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c00f59-7df4-403a-a97b-daa3dccc7d1b admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:32.079995 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58962c16-d0a1-4f3c-a4fa-b8ecd05cad6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:32.080690 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58962c16-d0a1-4f3c-a4fa-b8ecd05cad6d admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:32.080969 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58962c16-d0a1-4f3c-a4fa-b8ecd05cad6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:32.081242 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58962c16-d0a1-4f3c-a4fa-b8ecd05cad6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:32.082020 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58962c16-d0a1-4f3c-a4fa-b8ecd05cad6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:32.083589 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58962c16-d0a1-4f3c-a4fa-b8ecd05cad6d admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:32.094621 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1348/2721] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:41:32.098565 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08df045-7216-4ca2-9675-402684fca537 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:32.109067 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08df045-7216-4ca2-9675-402684fca537 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:32.113900 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef7d747-dd08-4771-9ee3-3f9f6aba844e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:32.114766 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef7d747-dd08-4771-9ee3-3f9f6aba844e admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:32.115049 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef7d747-dd08-4771-9ee3-3f9f6aba844e admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:32.115323 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef7d747-dd08-4771-9ee3-3f9f6aba844e admin admin] PATH_INFO: `/v3/projects/1a46f8b3333b41859d2735e9ca956f97/users/2e712448c5d44137849a7c62ffe98415/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:32.129679 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef7d747-dd08-4771-9ee3-3f9f6aba844e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a46f8b3333b41859d2735e9ca956f97, user_id=2e712448c5d44137849a7c62ffe98415, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:32.132362 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef7d747-dd08-4771-9ee3-3f9f6aba844e admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:32.201689 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1374/2722] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:41:32 2022] PUT /identity/v3/projects/1a46f8b3333b41859d2735e9ca956f97/users/2e712448c5d44137849a7c62ffe98415/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:32.205587 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58962c16-d0a1-4f3c-a4fa-b8ecd05cad6d admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:32.216261 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58962c16-d0a1-4f3c-a4fa-b8ecd05cad6d admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:32.270605 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef0816-87e1-46e5-bc61-c42fe93ce5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:32.271277 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef0816-87e1-46e5-bc61-c42fe93ce5f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:32.271555 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef0816-87e1-46e5-bc61-c42fe93ce5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:32.271855 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef0816-87e1-46e5-bc61-c42fe93ce5f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:32.273220 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef0816-87e1-46e5-bc61-c42fe93ce5f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:32.274785 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef0816-87e1-46e5-bc61-c42fe93ce5f6 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:32.284334 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1349/2723] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:41:32.289039 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef7d747-dd08-4771-9ee3-3f9f6aba844e admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:32.299254 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef7d747-dd08-4771-9ee3-3f9f6aba844e admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:32.303887 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb95fe8-5100-4d38-8a83-dc4be3feab12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:32.304759 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb95fe8-5100-4d38-8a83-dc4be3feab12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:32.305041 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb95fe8-5100-4d38-8a83-dc4be3feab12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:32.305320 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb95fe8-5100-4d38-8a83-dc4be3feab12 admin admin] PATH_INFO: `/v3/projects/1a46f8b3333b41859d2735e9ca956f97/users/2e712448c5d44137849a7c62ffe98415/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:32.307278 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb95fe8-5100-4d38-8a83-dc4be3feab12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a46f8b3333b41859d2735e9ca956f97, user_id=2e712448c5d44137849a7c62ffe98415, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:32.309927 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb95fe8-5100-4d38-8a83-dc4be3feab12 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:32.320202 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1375/2724] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:41:32 2022] PUT /identity/v3/projects/1a46f8b3333b41859d2735e9ca956f97/users/2e712448c5d44137849a7c62ffe98415/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:32.330290 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e27f05-db6c-4662-83e8-89ebc1124ba1 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:32.330599 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e27f05-db6c-4662-83e8-89ebc1124ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:32.330875 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e27f05-db6c-4662-83e8-89ebc1124ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:32.355173 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:32.358190 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-f1e27f05-db6c-4662-83e8-89ebc1124ba1 None None] MFA Rules not processed for user `2e712448c5d44137849a7c62ffe98415`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:32.359406 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-f1e27f05-db6c-4662-83e8-89ebc1124ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:32.426152 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1350/2725] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:32 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:32.436874 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb95fe8-5100-4d38-8a83-dc4be3feab12 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:32.438546 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-6fb95fe8-5100-4d38-8a83-dc4be3feab12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:32.494934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb95fe8-5100-4d38-8a83-dc4be3feab12 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:32.498616 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efad5214-9b98-4fcd-8863-52ca48284f78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:32.499248 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad5214-9b98-4fcd-8863-52ca48284f78 service cinder] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:32.499531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad5214-9b98-4fcd-8863-52ca48284f78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:32.499833 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad5214-9b98-4fcd-8863-52ca48284f78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:32.518792 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:32.555310 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efad5214-9b98-4fcd-8863-52ca48284f78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:32.556846 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efad5214-9b98-4fcd-8863-52ca48284f78 service cinder] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:32.566246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1376/2726] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:41:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:41:35.530646 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e27f05-db6c-4662-83e8-89ebc1124ba1 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:35.596315 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e27f05-db6c-4662-83e8-89ebc1124ba1 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:35.601094 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a8082c-9388-40ff-8954-f57acfdf0ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:35.602038 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8082c-9388-40ff-8954-f57acfdf0ca7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:35.602324 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8082c-9388-40ff-8954-f57acfdf0ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:35.602637 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8082c-9388-40ff-8954-f57acfdf0ca7 admin admin] PATH_INFO: `/v3/users/e4657f81cba64ff9a0385f46b75bd40a` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:35.603872 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a8082c-9388-40ff-8954-f57acfdf0ca7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4657f81cba64ff9a0385f46b75bd40a)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:35.605789 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a8082c-9388-40ff-8954-f57acfdf0ca7 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:35.760679 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-a3a8082c-9388-40ff-8954-f57acfdf0ca7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4657f81cba64ff9a0385f46b75bd40a'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:35.768550 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-a3a8082c-9388-40ff-8954-f57acfdf0ca7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4657f81cba64ff9a0385f46b75bd40a'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:35.781320 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-a3a8082c-9388-40ff-8954-f57acfdf0ca7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4657f81cba64ff9a0385f46b75bd40a'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:35.782182 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-a3a8082c-9388-40ff-8954-f57acfdf0ca7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4657f81cba64ff9a0385f46b75bd40a'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:35.859402 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-a3a8082c-9388-40ff-8954-f57acfdf0ca7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4657f81cba64ff9a0385f46b75bd40a'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:35.865983 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1351/2727] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:41:35 2022] DELETE /identity/v3/users/e4657f81cba64ff9a0385f46b75bd40a => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:41:35.870657 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c213981-ec58-47dd-b96c-72b5e7bc87c1 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:35.871138 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c213981-ec58-47dd-b96c-72b5e7bc87c1 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:35.871238 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c213981-ec58-47dd-b96c-72b5e7bc87c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:35.888027 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-6c213981-ec58-47dd-b96c-72b5e7bc87c1 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:35.889240 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-6c213981-ec58-47dd-b96c-72b5e7bc87c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:35.957165 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1377/2728] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:41:35.973152 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a8082c-9388-40ff-8954-f57acfdf0ca7 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:35.973152 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-a3a8082c-9388-40ff-8954-f57acfdf0ca7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:35.994653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:36.036050 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a8082c-9388-40ff-8954-f57acfdf0ca7 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:36.040446 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82554b72-a61d-4b90-82b6-bf221716b0a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:36.041195 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82554b72-a61d-4b90-82b6-bf221716b0a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:36.041457 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82554b72-a61d-4b90-82b6-bf221716b0a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:36.041732 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82554b72-a61d-4b90-82b6-bf221716b0a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:36.050734 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82554b72-a61d-4b90-82b6-bf221716b0a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:36.052244 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82554b72-a61d-4b90-82b6-bf221716b0a8 service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:36.061896 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1352/2729] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:41:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:41:36.444726 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c213981-ec58-47dd-b96c-72b5e7bc87c1 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:36.446824 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-6c213981-ec58-47dd-b96c-72b5e7bc87c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:36.463284 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c213981-ec58-47dd-b96c-72b5e7bc87c1 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:36.468157 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda15838-3cb4-4d08-a1ec-55759609cd50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:36.469064 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda15838-3cb4-4d08-a1ec-55759609cd50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:36.469418 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda15838-3cb4-4d08-a1ec-55759609cd50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:36.469602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda15838-3cb4-4d08-a1ec-55759609cd50 admin admin] PATH_INFO: `/v3/projects/cc35b009de744e7298b272f82377e8a8` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:36.470863 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda15838-3cb4-4d08-a1ec-55759609cd50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc35b009de744e7298b272f82377e8a8)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:36.472808 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda15838-3cb4-4d08-a1ec-55759609cd50 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:36.557664 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-fda15838-3cb4-4d08-a1ec-55759609cd50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc35b009de744e7298b272f82377e8a8', 'request_id': 'req-fda15838-3cb4-4d08-a1ec-55759609cd50'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:36.564140 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-fda15838-3cb4-4d08-a1ec-55759609cd50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc35b009de744e7298b272f82377e8a8', 'request_id': 'req-fda15838-3cb4-4d08-a1ec-55759609cd50'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:36.571097 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-fda15838-3cb4-4d08-a1ec-55759609cd50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc35b009de744e7298b272f82377e8a8', 'request_id': 'req-fda15838-3cb4-4d08-a1ec-55759609cd50'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:36.670029 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-fda15838-3cb4-4d08-a1ec-55759609cd50 admin admin] The token cache is being invalidate because project cc35b009de744e7298b272f82377e8a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:41:36.670319 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-fda15838-3cb4-4d08-a1ec-55759609cd50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:36.670959 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-fda15838-3cb4-4d08-a1ec-55759609cd50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:36.672572 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1378/2730] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:41:36 2022] DELETE /identity/v3/projects/cc35b009de744e7298b272f82377e8a8 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:36.676585 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82554b72-a61d-4b90-82b6-bf221716b0a8 service neutron] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:36.686675 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82554b72-a61d-4b90-82b6-bf221716b0a8 service neutron] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:36.744362 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab1de8-5831-4548-852b-f911263afc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:36.745229 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab1de8-5831-4548-852b-f911263afc48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:36.745511 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab1de8-5831-4548-852b-f911263afc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:36.745783 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab1de8-5831-4548-852b-f911263afc48 admin admin] PATH_INFO: `/v3/users/2e712448c5d44137849a7c62ffe98415` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:36.746957 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bab1de8-5831-4548-852b-f911263afc48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e712448c5d44137849a7c62ffe98415)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:36.749022 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bab1de8-5831-4548-852b-f911263afc48 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:36.925581 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-6bab1de8-5831-4548-852b-f911263afc48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e712448c5d44137849a7c62ffe98415'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:36.933366 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-6bab1de8-5831-4548-852b-f911263afc48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e712448c5d44137849a7c62ffe98415'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:36.945416 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-6bab1de8-5831-4548-852b-f911263afc48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e712448c5d44137849a7c62ffe98415'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:36.946154 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-6bab1de8-5831-4548-852b-f911263afc48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e712448c5d44137849a7c62ffe98415'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:37.032533 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-6bab1de8-5831-4548-852b-f911263afc48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e712448c5d44137849a7c62ffe98415'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:37.040210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1353/2731] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:41:36 2022] DELETE /identity/v3/users/2e712448c5d44137849a7c62ffe98415 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:37.453370 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda15838-3cb4-4d08-a1ec-55759609cd50 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:37.454952 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-fda15838-3cb4-4d08-a1ec-55759609cd50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:37.468819 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda15838-3cb4-4d08-a1ec-55759609cd50 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:37.510519 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:37.514550 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab1de8-5831-4548-852b-f911263afc48 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:37.518909 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-6bab1de8-5831-4548-852b-f911263afc48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:37.532965 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c74b39-4fe7-47cd-9de8-650e8461b9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:37.534544 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c74b39-4fe7-47cd-9de8-650e8461b9bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:37.534838 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c74b39-4fe7-47cd-9de8-650e8461b9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:37.541892 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c74b39-4fe7-47cd-9de8-650e8461b9bd admin admin] PATH_INFO: `/v3/users/5fe6848dd346440e97b91dd79610a874` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:37.541892 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c74b39-4fe7-47cd-9de8-650e8461b9bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fe6848dd346440e97b91dd79610a874)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:37.548195 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c74b39-4fe7-47cd-9de8-650e8461b9bd admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:37.576896 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:37.599556 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab1de8-5831-4548-852b-f911263afc48 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:37.608169 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40da0ee-06e7-4d1b-9b6d-4fd470238fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:37.609486 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40da0ee-06e7-4d1b-9b6d-4fd470238fe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:37.609753 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40da0ee-06e7-4d1b-9b6d-4fd470238fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:37.610032 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40da0ee-06e7-4d1b-9b6d-4fd470238fe4 admin admin] PATH_INFO: `/v3/projects/1a46f8b3333b41859d2735e9ca956f97` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:37.611300 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40da0ee-06e7-4d1b-9b6d-4fd470238fe4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a46f8b3333b41859d2735e9ca956f97)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:37.613647 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40da0ee-06e7-4d1b-9b6d-4fd470238fe4 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:37.753971 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-51c74b39-4fe7-47cd-9de8-650e8461b9bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fe6848dd346440e97b91dd79610a874'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:37.754374 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-b40da0ee-06e7-4d1b-9b6d-4fd470238fe4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a46f8b3333b41859d2735e9ca956f97', 'request_id': 'req-b40da0ee-06e7-4d1b-9b6d-4fd470238fe4'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:37.754831 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-51c74b39-4fe7-47cd-9de8-650e8461b9bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fe6848dd346440e97b91dd79610a874'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:37.761335 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-b40da0ee-06e7-4d1b-9b6d-4fd470238fe4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a46f8b3333b41859d2735e9ca956f97', 'request_id': 'req-b40da0ee-06e7-4d1b-9b6d-4fd470238fe4'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:37.762713 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-51c74b39-4fe7-47cd-9de8-650e8461b9bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fe6848dd346440e97b91dd79610a874'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:37.763410 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-51c74b39-4fe7-47cd-9de8-650e8461b9bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fe6848dd346440e97b91dd79610a874'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:37.767496 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-b40da0ee-06e7-4d1b-9b6d-4fd470238fe4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a46f8b3333b41859d2735e9ca956f97', 'request_id': 'req-b40da0ee-06e7-4d1b-9b6d-4fd470238fe4'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:37.775881 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-51c74b39-4fe7-47cd-9de8-650e8461b9bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fe6848dd346440e97b91dd79610a874'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:37.854522 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-b40da0ee-06e7-4d1b-9b6d-4fd470238fe4 admin admin] The token cache is being invalidate because project 1a46f8b3333b41859d2735e9ca956f97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:41:37.854728 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-b40da0ee-06e7-4d1b-9b6d-4fd470238fe4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:37.855387 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-b40da0ee-06e7-4d1b-9b6d-4fd470238fe4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:37.857022 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1354/2732] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:41:37 2022] DELETE /identity/v3/projects/1a46f8b3333b41859d2735e9ca956f97 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:41:37.861845 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fdb4ba-3fc6-4c98-86cf-a0d3e2ddd23c None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:37.862391 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fdb4ba-3fc6-4c98-86cf-a0d3e2ddd23c None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:37.862661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fdb4ba-3fc6-4c98-86cf-a0d3e2ddd23c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:37.880148 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-05fdb4ba-3fc6-4c98-86cf-a0d3e2ddd23c None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:37.884220 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-05fdb4ba-3fc6-4c98-86cf-a0d3e2ddd23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:37.935369 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1379/2733] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:41:37 2022] DELETE /identity/v3/users/5fe6848dd346440e97b91dd79610a874 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:37.939773 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca513890-fae0-4f78-b2eb-31050e2d095c None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:37.940063 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca513890-fae0-4f78-b2eb-31050e2d095c None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:37.940343 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca513890-fae0-4f78-b2eb-31050e2d095c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:37.940557 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1355/2734] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:37.949877 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19dffa-236b-4ea1-ba6b-c2b02af7fd3e None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:37.950061 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19dffa-236b-4ea1-ba6b-c2b02af7fd3e None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:37.950414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19dffa-236b-4ea1-ba6b-c2b02af7fd3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:37.964226 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-ca513890-fae0-4f78-b2eb-31050e2d095c None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:37.965428 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-ca513890-fae0-4f78-b2eb-31050e2d095c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:37.969884 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-2a19dffa-236b-4ea1-ba6b-c2b02af7fd3e None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:37.971132 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-2a19dffa-236b-4ea1-ba6b-c2b02af7fd3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:38.024823 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1380/2735] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:38.031092 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1356/2736] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:41:38.034727 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca513890-fae0-4f78-b2eb-31050e2d095c None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:38.036112 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-ca513890-fae0-4f78-b2eb-31050e2d095c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:38.039156 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19dffa-236b-4ea1-ba6b-c2b02af7fd3e None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:38.048209 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19dffa-236b-4ea1-ba6b-c2b02af7fd3e None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:38.053748 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb2b33-0a5c-4282-8173-1e587a94ab13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:38.059428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb2b33-0a5c-4282-8173-1e587a94ab13 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:38.059428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb2b33-0a5c-4282-8173-1e587a94ab13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:38.059428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb2b33-0a5c-4282-8173-1e587a94ab13 admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:38.059428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eb2b33-0a5c-4282-8173-1e587a94ab13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:41:38.059428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eb2b33-0a5c-4282-8173-1e587a94ab13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:38.059428 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eb2b33-0a5c-4282-8173-1e587a94ab13 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:38.069650 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1357/2737] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:41:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:41:38.074621 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb2b33-0a5c-4282-8173-1e587a94ab13 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:38.084776 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb2b33-0a5c-4282-8173-1e587a94ab13 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:38.089578 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e42a3f6-c755-4e80-9953-dee6d28d7607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:38.090221 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42a3f6-c755-4e80-9953-dee6d28d7607 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:38.090507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42a3f6-c755-4e80-9953-dee6d28d7607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:38.090785 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42a3f6-c755-4e80-9953-dee6d28d7607 admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:38.091483 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e42a3f6-c755-4e80-9953-dee6d28d7607 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:38.093494 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e42a3f6-c755-4e80-9953-dee6d28d7607 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:38.098231 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca513890-fae0-4f78-b2eb-31050e2d095c None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:38.102188 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c917016-a01f-4a16-8cac-af3711a5e7dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:38.102837 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c917016-a01f-4a16-8cac-af3711a5e7dc service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:38.103126 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c917016-a01f-4a16-8cac-af3711a5e7dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:38.103405 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c917016-a01f-4a16-8cac-af3711a5e7dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:38.112391 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:38.113427 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c917016-a01f-4a16-8cac-af3711a5e7dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:38.114898 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c917016-a01f-4a16-8cac-af3711a5e7dc service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:38.125244 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1381/2738] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:41:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:41:38.183689 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1358/2739] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:41:38 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:41:38.187970 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c917016-a01f-4a16-8cac-af3711a5e7dc service neutron] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:38.198267 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c917016-a01f-4a16-8cac-af3711a5e7dc service neutron] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:38.201977 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:38.254307 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc9dca-3e8b-46f1-a2ad-c2edef7c1328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:38.254976 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc9dca-3e8b-46f1-a2ad-c2edef7c1328 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:38.255261 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc9dca-3e8b-46f1-a2ad-c2edef7c1328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:38.255540 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc9dca-3e8b-46f1-a2ad-c2edef7c1328 admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:38.256251 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bc9dca-3e8b-46f1-a2ad-c2edef7c1328 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:38.258795 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bc9dca-3e8b-46f1-a2ad-c2edef7c1328 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:38.335711 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1382/2740] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:41:38 2022] POST /identity/v3/users => generated 293 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:41:38.339832 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e42a3f6-c755-4e80-9953-dee6d28d7607 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:38.363292 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e42a3f6-c755-4e80-9953-dee6d28d7607 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:38.369080 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2775b07a-2be2-4d09-a2c1-aba49e70d344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:38.369870 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2775b07a-2be2-4d09-a2c1-aba49e70d344 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:38.370096 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2775b07a-2be2-4d09-a2c1-aba49e70d344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:38.370393 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2775b07a-2be2-4d09-a2c1-aba49e70d344 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:38.371123 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2775b07a-2be2-4d09-a2c1-aba49e70d344 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:38.373006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2775b07a-2be2-4d09-a2c1-aba49e70d344 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:38.384532 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1359/2741] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:38 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:41:38.388741 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc9dca-3e8b-46f1-a2ad-c2edef7c1328 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:38.399765 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc9dca-3e8b-46f1-a2ad-c2edef7c1328 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:38.404766 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63612261-3130-48b4-8059-ae75331e6381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:38.405434 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63612261-3130-48b4-8059-ae75331e6381 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:38.405715 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63612261-3130-48b4-8059-ae75331e6381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:38.405992 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63612261-3130-48b4-8059-ae75331e6381 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:38.406726 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63612261-3130-48b4-8059-ae75331e6381 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:38.408404 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63612261-3130-48b4-8059-ae75331e6381 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:38.420084 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1383/2742] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:41:38.424399 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2775b07a-2be2-4d09-a2c1-aba49e70d344 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:38.436062 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2775b07a-2be2-4d09-a2c1-aba49e70d344 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:38.440907 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261d2f9-eeef-46dc-9ea7-fb8e5d6e1462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:38.441919 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261d2f9-eeef-46dc-9ea7-fb8e5d6e1462 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:38.442208 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261d2f9-eeef-46dc-9ea7-fb8e5d6e1462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:38.442513 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261d2f9-eeef-46dc-9ea7-fb8e5d6e1462 admin admin] PATH_INFO: `/v3/projects/0b2770b1b56d4f0ea8c3c1759a8e504f/users/975faecf991a41f4b116537b39e04d3c/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:38.459667 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1261d2f9-eeef-46dc-9ea7-fb8e5d6e1462 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b2770b1b56d4f0ea8c3c1759a8e504f, user_id=975faecf991a41f4b116537b39e04d3c, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:38.462345 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1261d2f9-eeef-46dc-9ea7-fb8e5d6e1462 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:38.559330 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1360/2743] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:41:38 2022] PUT /identity/v3/projects/0b2770b1b56d4f0ea8c3c1759a8e504f/users/975faecf991a41f4b116537b39e04d3c/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:38.564301 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc6c21-8997-4b55-ab4e-2e66f374f8cf None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:38.564764 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc6c21-8997-4b55-ab4e-2e66f374f8cf None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:38.564974 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc6c21-8997-4b55-ab4e-2e66f374f8cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:38.582826 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261d2f9-eeef-46dc-9ea7-fb8e5d6e1462 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:38.584948 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-88fc6c21-8997-4b55-ab4e-2e66f374f8cf None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:38.585190 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-1261d2f9-eeef-46dc-9ea7-fb8e5d6e1462 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:38.586325 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-88fc6c21-8997-4b55-ab4e-2e66f374f8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:38.662379 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1384/2744] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:38.673428 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc6c21-8997-4b55-ab4e-2e66f374f8cf None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:38.676060 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-88fc6c21-8997-4b55-ab4e-2e66f374f8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:38.685194 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261d2f9-eeef-46dc-9ea7-fb8e5d6e1462 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:38.690269 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6ad28-7563-452d-8e02-65860a54404d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:38.691147 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6ad28-7563-452d-8e02-65860a54404d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:38.691445 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6ad28-7563-452d-8e02-65860a54404d admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:38.691715 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6ad28-7563-452d-8e02-65860a54404d admin admin] PATH_INFO: `/v3/projects/74aed37262f04c5b8cd552b2a4edd0ea` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:38.692925 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b6ad28-7563-452d-8e02-65860a54404d admin admin] RBAC: Authorizing `identity:delete_project(project_id=74aed37262f04c5b8cd552b2a4edd0ea)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:38.694871 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b6ad28-7563-452d-8e02-65860a54404d admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:38.749266 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc6c21-8997-4b55-ab4e-2e66f374f8cf None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:38.752878 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce91bc-b0c2-4598-b735-8a13f35a5802 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:38.753555 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce91bc-b0c2-4598-b735-8a13f35a5802 service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:38.753825 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce91bc-b0c2-4598-b735-8a13f35a5802 service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:38.754096 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce91bc-b0c2-4598-b735-8a13f35a5802 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:38.763292 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ce91bc-b0c2-4598-b735-8a13f35a5802 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:38.764748 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ce91bc-b0c2-4598-b735-8a13f35a5802 service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:38.774574 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1385/2745] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:41:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 15:41:38.861147 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-99b6ad28-7563-452d-8e02-65860a54404d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74aed37262f04c5b8cd552b2a4edd0ea', 'request_id': 'req-99b6ad28-7563-452d-8e02-65860a54404d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:38.867495 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-99b6ad28-7563-452d-8e02-65860a54404d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74aed37262f04c5b8cd552b2a4edd0ea', 'request_id': 'req-99b6ad28-7563-452d-8e02-65860a54404d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:38.874203 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-99b6ad28-7563-452d-8e02-65860a54404d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74aed37262f04c5b8cd552b2a4edd0ea', 'request_id': 'req-99b6ad28-7563-452d-8e02-65860a54404d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:38.964009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-99b6ad28-7563-452d-8e02-65860a54404d admin admin] The token cache is being invalidate because project 74aed37262f04c5b8cd552b2a4edd0ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:41:38.964311 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-99b6ad28-7563-452d-8e02-65860a54404d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:38.965156 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-99b6ad28-7563-452d-8e02-65860a54404d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:38.967181 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1361/2746] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:41:38 2022] DELETE /identity/v3/projects/74aed37262f04c5b8cd552b2a4edd0ea => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:41:38.971171 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce91bc-b0c2-4598-b735-8a13f35a5802 service neutron] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:38.981867 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce91bc-b0c2-4598-b735-8a13f35a5802 service neutron] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:39.008157 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:39.041372 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061591d-3829-47ee-bbdb-f82d866b4c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:39.042543 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061591d-3829-47ee-bbdb-f82d866b4c0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:39.043552 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061591d-3829-47ee-bbdb-f82d866b4c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:39.044184 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061591d-3829-47ee-bbdb-f82d866b4c0c admin admin] PATH_INFO: `/v3/users/19f27998156e413cac45c16780dafd7a` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:39.048055 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b061591d-3829-47ee-bbdb-f82d866b4c0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=19f27998156e413cac45c16780dafd7a)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:39.048481 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b061591d-3829-47ee-bbdb-f82d866b4c0c admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:39.272558 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-b061591d-3829-47ee-bbdb-f82d866b4c0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19f27998156e413cac45c16780dafd7a'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:39.273823 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-b061591d-3829-47ee-bbdb-f82d866b4c0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19f27998156e413cac45c16780dafd7a'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:39.283395 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-b061591d-3829-47ee-bbdb-f82d866b4c0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19f27998156e413cac45c16780dafd7a'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:39.297137 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-b061591d-3829-47ee-bbdb-f82d866b4c0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19f27998156e413cac45c16780dafd7a'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:39.297474 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-b061591d-3829-47ee-bbdb-f82d866b4c0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19f27998156e413cac45c16780dafd7a'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:39.431306 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1386/2747] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:41:38 2022] DELETE /identity/v3/users/19f27998156e413cac45c16780dafd7a => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:39.958618 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6ad28-7563-452d-8e02-65860a54404d admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:39.960233 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-99b6ad28-7563-452d-8e02-65860a54404d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:39.974183 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6ad28-7563-452d-8e02-65860a54404d admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:40.005525 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:40.032098 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d82b2b-2867-40bb-b623-1000e99b050d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:40.032883 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d82b2b-2867-40bb-b623-1000e99b050d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:40.033161 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d82b2b-2867-40bb-b623-1000e99b050d admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:40.033430 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d82b2b-2867-40bb-b623-1000e99b050d admin admin] PATH_INFO: `/v3/projects/b69194410fb547a99d528b1fb3b27e73` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:40.034588 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d82b2b-2867-40bb-b623-1000e99b050d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b69194410fb547a99d528b1fb3b27e73)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:40.036599 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d82b2b-2867-40bb-b623-1000e99b050d admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:40.183159 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-82d82b2b-2867-40bb-b623-1000e99b050d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b69194410fb547a99d528b1fb3b27e73', 'request_id': 'req-82d82b2b-2867-40bb-b623-1000e99b050d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:40.189189 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-82d82b2b-2867-40bb-b623-1000e99b050d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b69194410fb547a99d528b1fb3b27e73', 'request_id': 'req-82d82b2b-2867-40bb-b623-1000e99b050d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:40.194060 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061591d-3829-47ee-bbdb-f82d866b4c0c admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:40.195917 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-82d82b2b-2867-40bb-b623-1000e99b050d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b69194410fb547a99d528b1fb3b27e73', 'request_id': 'req-82d82b2b-2867-40bb-b623-1000e99b050d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:40.200211 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-b061591d-3829-47ee-bbdb-f82d866b4c0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:40.272273 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061591d-3829-47ee-bbdb-f82d866b4c0c admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:40.322685 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b28f0-40bb-4221-8843-dc83bb6756c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:40.323338 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b28f0-40bb-4221-8843-dc83bb6756c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:40.323648 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b28f0-40bb-4221-8843-dc83bb6756c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:40.323927 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b28f0-40bb-4221-8843-dc83bb6756c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:40.325716 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-296b28f0-40bb-4221-8843-dc83bb6756c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:40.346578 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:40.349967 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-82d82b2b-2867-40bb-b623-1000e99b050d admin admin] The token cache is being invalidate because project b69194410fb547a99d528b1fb3b27e73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:41:40.349967 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-82d82b2b-2867-40bb-b623-1000e99b050d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:40.353455 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-82d82b2b-2867-40bb-b623-1000e99b050d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:40.357995 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1362/2748] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:41:39 2022] DELETE /identity/v3/projects/b69194410fb547a99d528b1fb3b27e73 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:41:40.366091 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e18b06a-bde8-4fa5-b4ea-e688c57827eb None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:40.368936 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e18b06a-bde8-4fa5-b4ea-e688c57827eb None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:40.369513 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e18b06a-bde8-4fa5-b4ea-e688c57827eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:40.397131 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296b28f0-40bb-4221-8843-dc83bb6756c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:40.397675 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-8e18b06a-bde8-4fa5-b4ea-e688c57827eb None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:40.399158 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296b28f0-40bb-4221-8843-dc83bb6756c6 service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:40.400266 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-8e18b06a-bde8-4fa5-b4ea-e688c57827eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:40.411999 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1387/2749] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:41:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 15:41:40.470424 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1363/2750] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:41:40.480195 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103a0be-ddd2-4adb-b375-554f9a178810 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:40.480486 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103a0be-ddd2-4adb-b375-554f9a178810 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:40.480764 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103a0be-ddd2-4adb-b375-554f9a178810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:40.498739 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-6103a0be-ddd2-4adb-b375-554f9a178810 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:40.500010 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-6103a0be-ddd2-4adb-b375-554f9a178810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:40.509686 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1388/2751] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:40.514049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e18b06a-bde8-4fa5-b4ea-e688c57827eb None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:40.525689 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e18b06a-bde8-4fa5-b4ea-e688c57827eb None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:40.531441 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959cf3e-0b1f-4b98-b75f-036f04c9ae7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:40.532152 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959cf3e-0b1f-4b98-b75f-036f04c9ae7b admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:40.532489 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959cf3e-0b1f-4b98-b75f-036f04c9ae7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:40.533167 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959cf3e-0b1f-4b98-b75f-036f04c9ae7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:40.533911 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2959cf3e-0b1f-4b98-b75f-036f04c9ae7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:41:40.534276 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2959cf3e-0b1f-4b98-b75f-036f04c9ae7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:40.537816 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2959cf3e-0b1f-4b98-b75f-036f04c9ae7b admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:40.557015 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103a0be-ddd2-4adb-b375-554f9a178810 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:40.559761 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1364/2752] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 15:41:40.570252 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103a0be-ddd2-4adb-b375-554f9a178810 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:40.575342 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e08c6-d859-4f5d-83c6-da155dce4e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:40.576050 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e08c6-d859-4f5d-83c6-da155dce4e16 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:40.576334 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e08c6-d859-4f5d-83c6-da155dce4e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:40.576612 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e08c6-d859-4f5d-83c6-da155dce4e16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:40.577315 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6e08c6-d859-4f5d-83c6-da155dce4e16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:40.581149 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6e08c6-d859-4f5d-83c6-da155dce4e16 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:40.694195 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1389/2753] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:41:40 2022] POST /identity/v3/projects => generated 387 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:41:40.695878 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959cf3e-0b1f-4b98-b75f-036f04c9ae7b admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:40.707830 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959cf3e-0b1f-4b98-b75f-036f04c9ae7b admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:40.805176 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d041537e-8e36-4e98-a53d-fd50a92e8902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:40.807230 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041537e-8e36-4e98-a53d-fd50a92e8902 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:40.807467 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041537e-8e36-4e98-a53d-fd50a92e8902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:40.811469 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041537e-8e36-4e98-a53d-fd50a92e8902 admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:40.811469 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d041537e-8e36-4e98-a53d-fd50a92e8902 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:40.814285 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d041537e-8e36-4e98-a53d-fd50a92e8902 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:40.969175 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1365/2754] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:41:40 2022] POST /identity/v3/users => generated 294 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:41:40.973026 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e08c6-d859-4f5d-83c6-da155dce4e16 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:40.986497 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e08c6-d859-4f5d-83c6-da155dce4e16 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:41.005005 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd577fd2-6cf8-4074-bd26-1376dbe64f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:41.005785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd577fd2-6cf8-4074-bd26-1376dbe64f7d admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:41.017768 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd577fd2-6cf8-4074-bd26-1376dbe64f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:41.018368 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd577fd2-6cf8-4074-bd26-1376dbe64f7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:41.019154 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd577fd2-6cf8-4074-bd26-1376dbe64f7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:41.022746 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd577fd2-6cf8-4074-bd26-1376dbe64f7d admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:41.036778 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1390/2755] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:41:41.041665 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d041537e-8e36-4e98-a53d-fd50a92e8902 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:41.066718 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:41.072370 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d041537e-8e36-4e98-a53d-fd50a92e8902 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:41.082885 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c421e46-03a0-4ee4-ab25-002e886104b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:41.089720 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c421e46-03a0-4ee4-ab25-002e886104b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:41.089720 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c421e46-03a0-4ee4-ab25-002e886104b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:41.089720 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c421e46-03a0-4ee4-ab25-002e886104b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:41.089720 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c421e46-03a0-4ee4-ab25-002e886104b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:41.092290 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c421e46-03a0-4ee4-ab25-002e886104b8 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:41.112642 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1366/2756] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:41:41.113609 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd577fd2-6cf8-4074-bd26-1376dbe64f7d admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:41.127156 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd577fd2-6cf8-4074-bd26-1376dbe64f7d admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:41.134688 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d28a83-658e-4deb-a4bb-f77a9452b36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:41.136683 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d28a83-658e-4deb-a4bb-f77a9452b36c admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:41.137137 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d28a83-658e-4deb-a4bb-f77a9452b36c admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:41.138003 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d28a83-658e-4deb-a4bb-f77a9452b36c admin admin] PATH_INFO: `/v3/projects/1e136cadea2d438b81f7a8ab29d47b05/users/a57ba4e5efaa4ae7817ac46359aa9d6c/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:41.156986 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d28a83-658e-4deb-a4bb-f77a9452b36c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e136cadea2d438b81f7a8ab29d47b05, user_id=a57ba4e5efaa4ae7817ac46359aa9d6c, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:41.160313 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d28a83-658e-4deb-a4bb-f77a9452b36c admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:41.263190 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1391/2757] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:41:41 2022] PUT /identity/v3/projects/1e136cadea2d438b81f7a8ab29d47b05/users/a57ba4e5efaa4ae7817ac46359aa9d6c/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:41:41.274551 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f5993-a210-482a-b119-e5aa719404e4 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:41.274843 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f5993-a210-482a-b119-e5aa719404e4 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:41.275117 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f5993-a210-482a-b119-e5aa719404e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:41.325827 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-495f5993-a210-482a-b119-e5aa719404e4 None None] MFA Rules not processed for user `a57ba4e5efaa4ae7817ac46359aa9d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:41.327285 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-495f5993-a210-482a-b119-e5aa719404e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:41.404008 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1367/2758] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:41.410444 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d28a83-658e-4deb-a4bb-f77a9452b36c admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:41.493995 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d28a83-658e-4deb-a4bb-f77a9452b36c admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:41.501468 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc53093-e910-46bc-891c-c2f92b92e415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:41.502275 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc53093-e910-46bc-891c-c2f92b92e415 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:41.505268 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc53093-e910-46bc-891c-c2f92b92e415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:41.505268 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc53093-e910-46bc-891c-c2f92b92e415 admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:41.505268 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc53093-e910-46bc-891c-c2f92b92e415 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:41.506553 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc53093-e910-46bc-891c-c2f92b92e415 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:41.609863 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1392/2759] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:41:41 2022] POST /identity/v3/projects => generated 387 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:41:41.610973 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f5993-a210-482a-b119-e5aa719404e4 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:41.624512 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f5993-a210-482a-b119-e5aa719404e4 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:41.704565 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b7f2d-df59-461e-b011-6f640bc172a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:41.705451 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b7f2d-df59-461e-b011-6f640bc172a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:41.705832 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b7f2d-df59-461e-b011-6f640bc172a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:41.706207 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b7f2d-df59-461e-b011-6f640bc172a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:41.707092 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0b7f2d-df59-461e-b011-6f640bc172a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:41.709169 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0b7f2d-df59-461e-b011-6f640bc172a0 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:41.836251 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1368/2760] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:41:41 2022] POST /identity/v3/users => generated 300 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:41:41.838167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc53093-e910-46bc-891c-c2f92b92e415 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:41.849666 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc53093-e910-46bc-891c-c2f92b92e415 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:41.855100 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba16e9e-1fc8-436b-bdbe-f4e2571a59b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:41.855814 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba16e9e-1fc8-436b-bdbe-f4e2571a59b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:41.856099 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba16e9e-1fc8-436b-bdbe-f4e2571a59b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:41.856376 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba16e9e-1fc8-436b-bdbe-f4e2571a59b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:41.857114 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba16e9e-1fc8-436b-bdbe-f4e2571a59b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:41.858734 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba16e9e-1fc8-436b-bdbe-f4e2571a59b0 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:41.876178 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1393/2761] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:41:41.884214 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b7f2d-df59-461e-b011-6f640bc172a0 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:41.898732 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b7f2d-df59-461e-b011-6f640bc172a0 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:41.908144 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eceb9e-f9a9-4337-babc-7f85d007bbbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:41.909057 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eceb9e-f9a9-4337-babc-7f85d007bbbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:41.909342 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eceb9e-f9a9-4337-babc-7f85d007bbbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:41.909617 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eceb9e-f9a9-4337-babc-7f85d007bbbe admin admin] PATH_INFO: `/v3/projects/449e15f8dbad4fdb90b2f2dfc76e7889/users/6162eb6bbee34972aa93e508688ee8fc/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:41.934155 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eceb9e-f9a9-4337-babc-7f85d007bbbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=449e15f8dbad4fdb90b2f2dfc76e7889, user_id=6162eb6bbee34972aa93e508688ee8fc, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:41.936868 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eceb9e-f9a9-4337-babc-7f85d007bbbe admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:42.053113 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1369/2762] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:41:41 2022] PUT /identity/v3/projects/449e15f8dbad4fdb90b2f2dfc76e7889/users/6162eb6bbee34972aa93e508688ee8fc/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:42.057160 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba16e9e-1fc8-436b-bdbe-f4e2571a59b0 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:42.070877 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba16e9e-1fc8-436b-bdbe-f4e2571a59b0 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:42.091247 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c268fc-2e8a-4e54-bdc5-23f939c70dbf None None] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:42.092453 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c268fc-2e8a-4e54-bdc5-23f939c70dbf None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:42.092975 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c268fc-2e8a-4e54-bdc5-23f939c70dbf None None] PATH_INFO: `/` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:42.095354 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1370/2763] 10.222.0.25 () {54 vars in 921 bytes} [Thu Jul 7 15:41:42 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 15:41:42.113706 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:42.145499 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d368f8-b40a-466c-883c-f25a8f42864a None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:42.145791 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d368f8-b40a-466c-883c-f25a8f42864a None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:42.146127 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d368f8-b40a-466c-883c-f25a8f42864a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:42.166903 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf64873-5e57-4557-a329-af3084e5191d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:42.168143 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf64873-5e57-4557-a329-af3084e5191d admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:42.168501 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf64873-5e57-4557-a329-af3084e5191d admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:42.168844 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf64873-5e57-4557-a329-af3084e5191d admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:42.169754 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf64873-5e57-4557-a329-af3084e5191d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:42.173266 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf64873-5e57-4557-a329-af3084e5191d admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:42.187630 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1394/2764] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:42 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:41:42.191961 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf64873-5e57-4557-a329-af3084e5191d admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:42.202355 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-38d368f8-b40a-466c-883c-f25a8f42864a None None] MFA Rules not processed for user `755c2782cfb042f89950b8fc862e4fa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:42.203870 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-38d368f8-b40a-466c-883c-f25a8f42864a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:42.207057 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf64873-5e57-4557-a329-af3084e5191d admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:42.208930 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:42.214498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36a1a7-9a33-4ed5-b57b-99f7753290da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:42.215683 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36a1a7-9a33-4ed5-b57b-99f7753290da admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:42.216118 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36a1a7-9a33-4ed5-b57b-99f7753290da admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:42.216503 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36a1a7-9a33-4ed5-b57b-99f7753290da admin admin] PATH_INFO: `/v3/projects/449e15f8dbad4fdb90b2f2dfc76e7889/users/6162eb6bbee34972aa93e508688ee8fc/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:42.219511 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36a1a7-9a33-4ed5-b57b-99f7753290da admin admin] RBAC: Authorizing `identity:create_grant(project_id=449e15f8dbad4fdb90b2f2dfc76e7889, user_id=6162eb6bbee34972aa93e508688ee8fc, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:42.223399 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36a1a7-9a33-4ed5-b57b-99f7753290da admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:42.238826 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1395/2765] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:41:42 2022] PUT /identity/v3/projects/449e15f8dbad4fdb90b2f2dfc76e7889/users/6162eb6bbee34972aa93e508688ee8fc/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:42.253533 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ad70f-fde6-4fe4-a0b1-b5245346c8d0 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:42.254124 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ad70f-fde6-4fe4-a0b1-b5245346c8d0 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:42.254504 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ad70f-fde6-4fe4-a0b1-b5245346c8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:42.316171 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-272ad70f-fde6-4fe4-a0b1-b5245346c8d0 None None] MFA Rules not processed for user `6162eb6bbee34972aa93e508688ee8fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:42.319233 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-272ad70f-fde6-4fe4-a0b1-b5245346c8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:42.323697 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1371/2766] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Jul 7 15:41:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:41:42.402026 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d368f8-b40a-466c-883c-f25a8f42864a None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:42.404124 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-38d368f8-b40a-466c-883c-f25a8f42864a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:42.437134 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d368f8-b40a-466c-883c-f25a8f42864a None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:42.437930 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1396/2767] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:42 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:42.447401 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ad70f-fde6-4fe4-a0b1-b5245346c8d0 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:42.448891 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-272ad70f-fde6-4fe4-a0b1-b5245346c8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:42.603327 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ff666-993f-4f0c-8c4c-2912f6d2c3f8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '755c2782cfb042f89950b8fc862e4fa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:42.604432 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ff666-993f-4f0c-8c4c-2912f6d2c3f8 service placement] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:42.604822 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ff666-993f-4f0c-8c4c-2912f6d2c3f8 service placement] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:42.605225 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ff666-993f-4f0c-8c4c-2912f6d2c3f8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:42.618783 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ad70f-fde6-4fe4-a0b1-b5245346c8d0 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:42.627355 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771ff666-993f-4f0c-8c4c-2912f6d2c3f8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:42.634516 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771ff666-993f-4f0c-8c4c-2912f6d2c3f8 service placement] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:42.636486 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51b7ee-774a-43ce-93d6-b4745718e711 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:42.637117 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51b7ee-774a-43ce-93d6-b4745718e711 service cinder] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:42.637400 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51b7ee-774a-43ce-93d6-b4745718e711 service cinder] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:42.637674 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51b7ee-774a-43ce-93d6-b4745718e711 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:42.651730 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d51b7ee-774a-43ce-93d6-b4745718e711 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:42.653402 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d51b7ee-774a-43ce-93d6-b4745718e711 service cinder] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:42.663310 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1372/2768] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:41:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 15:41:42.673523 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1397/2769] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:41:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 15:41:43.862676 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ff666-993f-4f0c-8c4c-2912f6d2c3f8 service placement] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:43.864692 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-771ff666-993f-4f0c-8c4c-2912f6d2c3f8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:43.954125 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ff666-993f-4f0c-8c4c-2912f6d2c3f8 service placement] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:43.958959 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21ea0f-3ba6-42ca-9e89-b35a8208d3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:43.962780 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21ea0f-3ba6-42ca-9e89-b35a8208d3d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:43.962780 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21ea0f-3ba6-42ca-9e89-b35a8208d3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:43.962780 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21ea0f-3ba6-42ca-9e89-b35a8208d3d7 admin admin] PATH_INFO: `/v3/users/a7e54f97003c4818a87d6c49bd83614d` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:43.962780 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21ea0f-3ba6-42ca-9e89-b35a8208d3d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7e54f97003c4818a87d6c49bd83614d)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:43.964706 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21ea0f-3ba6-42ca-9e89-b35a8208d3d7 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:44.233591 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51b7ee-774a-43ce-93d6-b4745718e711 service cinder] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:44.247847 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:44.256071 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51b7ee-774a-43ce-93d6-b4745718e711 service cinder] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:44.260971 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-ae21ea0f-3ba6-42ca-9e89-b35a8208d3d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7e54f97003c4818a87d6c49bd83614d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:44.269371 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b72e91-f894-4220-bbf2-41763a35d1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:44.271583 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-ae21ea0f-3ba6-42ca-9e89-b35a8208d3d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7e54f97003c4818a87d6c49bd83614d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:44.272182 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b72e91-f894-4220-bbf2-41763a35d1a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:44.272619 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b72e91-f894-4220-bbf2-41763a35d1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:44.273155 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b72e91-f894-4220-bbf2-41763a35d1a9 admin admin] PATH_INFO: `/v3/users/a57ba4e5efaa4ae7817ac46359aa9d6c` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:44.276340 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b72e91-f894-4220-bbf2-41763a35d1a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a57ba4e5efaa4ae7817ac46359aa9d6c)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:44.280449 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b72e91-f894-4220-bbf2-41763a35d1a9 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:44.288130 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-ae21ea0f-3ba6-42ca-9e89-b35a8208d3d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7e54f97003c4818a87d6c49bd83614d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:44.289090 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-ae21ea0f-3ba6-42ca-9e89-b35a8208d3d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7e54f97003c4818a87d6c49bd83614d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:44.408863 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-ae21ea0f-3ba6-42ca-9e89-b35a8208d3d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7e54f97003c4818a87d6c49bd83614d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:44.417059 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1373/2770] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:41:43 2022] DELETE /identity/v3/users/a7e54f97003c4818a87d6c49bd83614d => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:44.606001 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:44.621936 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-a9b72e91-f894-4220-bbf2-41763a35d1a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a57ba4e5efaa4ae7817ac46359aa9d6c'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:44.623042 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-a9b72e91-f894-4220-bbf2-41763a35d1a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a57ba4e5efaa4ae7817ac46359aa9d6c'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:44.632010 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-a9b72e91-f894-4220-bbf2-41763a35d1a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a57ba4e5efaa4ae7817ac46359aa9d6c'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:44.633987 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-a9b72e91-f894-4220-bbf2-41763a35d1a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a57ba4e5efaa4ae7817ac46359aa9d6c'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:44.654248 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-a9b72e91-f894-4220-bbf2-41763a35d1a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a57ba4e5efaa4ae7817ac46359aa9d6c'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:44.763410 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1398/2771] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:41:44 2022] DELETE /identity/v3/users/a57ba4e5efaa4ae7817ac46359aa9d6c => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:44.767897 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20194eb5-93d3-493f-958c-23522641364e None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:44.768189 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20194eb5-93d3-493f-958c-23522641364e None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:44.768464 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20194eb5-93d3-493f-958c-23522641364e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:44.790520 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-20194eb5-93d3-493f-958c-23522641364e None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:44.794414 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-20194eb5-93d3-493f-958c-23522641364e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:44.795876 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b72e91-f894-4220-bbf2-41763a35d1a9 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:44.798282 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-a9b72e91-f894-4220-bbf2-41763a35d1a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:44.928439 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b72e91-f894-4220-bbf2-41763a35d1a9 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:44.937312 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca75c572-8488-43ce-95a6-1ba35a353419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:44.938217 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75c572-8488-43ce-95a6-1ba35a353419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:44.938501 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75c572-8488-43ce-95a6-1ba35a353419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:44.938777 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75c572-8488-43ce-95a6-1ba35a353419 admin admin] PATH_INFO: `/v3/projects/e01375189eb04e17bacc1c3892b86b9e` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:44.938964 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1374/2772] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:41:44.941093 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca75c572-8488-43ce-95a6-1ba35a353419 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e01375189eb04e17bacc1c3892b86b9e)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:44.949666 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca75c572-8488-43ce-95a6-1ba35a353419 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:45.015009 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20194eb5-93d3-493f-958c-23522641364e None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:45.016550 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-20194eb5-93d3-493f-958c-23522641364e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:45.131924 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-ca75c572-8488-43ce-95a6-1ba35a353419 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e01375189eb04e17bacc1c3892b86b9e', 'request_id': 'req-ca75c572-8488-43ce-95a6-1ba35a353419'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:45.156556 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-ca75c572-8488-43ce-95a6-1ba35a353419 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e01375189eb04e17bacc1c3892b86b9e', 'request_id': 'req-ca75c572-8488-43ce-95a6-1ba35a353419'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:45.158923 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20194eb5-93d3-493f-958c-23522641364e None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:45.176279 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-ca75c572-8488-43ce-95a6-1ba35a353419 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e01375189eb04e17bacc1c3892b86b9e', 'request_id': 'req-ca75c572-8488-43ce-95a6-1ba35a353419'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:45.250329 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b793994b-c1ae-4c22-8d21-db0c29a45953 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:45.251962 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793994b-c1ae-4c22-8d21-db0c29a45953 service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:45.252709 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793994b-c1ae-4c22-8d21-db0c29a45953 service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:45.253775 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793994b-c1ae-4c22-8d21-db0c29a45953 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:45.255893 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-ca75c572-8488-43ce-95a6-1ba35a353419 admin admin] The token cache is being invalidate because project e01375189eb04e17bacc1c3892b86b9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:41:45.257123 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-ca75c572-8488-43ce-95a6-1ba35a353419 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:45.258010 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-ca75c572-8488-43ce-95a6-1ba35a353419 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:45.259727 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1399/2773] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:41:44 2022] DELETE /identity/v3/projects/e01375189eb04e17bacc1c3892b86b9e => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:45.263935 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca75c572-8488-43ce-95a6-1ba35a353419 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:45.266621 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-ca75c572-8488-43ce-95a6-1ba35a353419 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:45.307458 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:45.385698 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b793994b-c1ae-4c22-8d21-db0c29a45953 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:45.388597 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b793994b-c1ae-4c22-8d21-db0c29a45953 service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:45.389422 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca75c572-8488-43ce-95a6-1ba35a353419 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:45.403480 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1375/2774] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:41:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 15:41:45.444273 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af18d0e8-ca76-47a3-bba1-a9f3a6efae00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:45.445663 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af18d0e8-ca76-47a3-bba1-a9f3a6efae00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:45.446404 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af18d0e8-ca76-47a3-bba1-a9f3a6efae00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:45.446987 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af18d0e8-ca76-47a3-bba1-a9f3a6efae00 admin admin] PATH_INFO: `/v3/users/6f549d6a5831404ea646a4129ee1d4ab` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:45.450377 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af18d0e8-ca76-47a3-bba1-a9f3a6efae00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f549d6a5831404ea646a4129ee1d4ab)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:45.453449 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af18d0e8-ca76-47a3-bba1-a9f3a6efae00 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:45.741381 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:45.820015 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-af18d0e8-ca76-47a3-bba1-a9f3a6efae00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f549d6a5831404ea646a4129ee1d4ab'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:45.821526 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-af18d0e8-ca76-47a3-bba1-a9f3a6efae00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f549d6a5831404ea646a4129ee1d4ab'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:45.832353 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-af18d0e8-ca76-47a3-bba1-a9f3a6efae00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f549d6a5831404ea646a4129ee1d4ab'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:45.833362 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-af18d0e8-ca76-47a3-bba1-a9f3a6efae00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f549d6a5831404ea646a4129ee1d4ab'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:45.865565 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-af18d0e8-ca76-47a3-bba1-a9f3a6efae00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f549d6a5831404ea646a4129ee1d4ab'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:45.980970 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1400/2775] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:41:45 2022] DELETE /identity/v3/users/6f549d6a5831404ea646a4129ee1d4ab => generated 0 bytes in 718 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:41:46.165575 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b793994b-c1ae-4c22-8d21-db0c29a45953 service neutron] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:46.169238 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-b793994b-c1ae-4c22-8d21-db0c29a45953 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:46.205053 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b793994b-c1ae-4c22-8d21-db0c29a45953 service neutron] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:46.297926 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d34cf-05ad-4662-ad87-02bce894fa4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:46.298917 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d34cf-05ad-4662-ad87-02bce894fa4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:46.299316 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d34cf-05ad-4662-ad87-02bce894fa4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:46.299723 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d34cf-05ad-4662-ad87-02bce894fa4e admin admin] PATH_INFO: `/v3/projects/1e136cadea2d438b81f7a8ab29d47b05` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:46.301142 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69d34cf-05ad-4662-ad87-02bce894fa4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e136cadea2d438b81f7a8ab29d47b05)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:46.303144 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69d34cf-05ad-4662-ad87-02bce894fa4e admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:46.348410 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3508110-0062-47ee-ab00-9d5fbc254d80 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:46.348935 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3508110-0062-47ee-ab00-9d5fbc254d80 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:46.351939 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3508110-0062-47ee-ab00-9d5fbc254d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:46.398027 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-c3508110-0062-47ee-ab00-9d5fbc254d80 None None] MFA Rules not processed for user `975faecf991a41f4b116537b39e04d3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:46.399853 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-c3508110-0062-47ee-ab00-9d5fbc254d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:46.476347 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1401/2776] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:46.477715 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3508110-0062-47ee-ab00-9d5fbc254d80 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:46.479115 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-c3508110-0062-47ee-ab00-9d5fbc254d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:46.493626 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3508110-0062-47ee-ab00-9d5fbc254d80 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:46.498177 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976714a7-f405-4ca9-bb4d-87f6ef25a4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:46.499006 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976714a7-f405-4ca9-bb4d-87f6ef25a4ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:46.499292 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976714a7-f405-4ca9-bb4d-87f6ef25a4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:46.499600 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976714a7-f405-4ca9-bb4d-87f6ef25a4ef admin admin] PATH_INFO: `/v3/projects/2818f02432344270b2ae3c1212432e01` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:46.500776 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976714a7-f405-4ca9-bb4d-87f6ef25a4ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=2818f02432344270b2ae3c1212432e01)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:46.502887 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976714a7-f405-4ca9-bb4d-87f6ef25a4ef admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:46.573884 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-f69d34cf-05ad-4662-ad87-02bce894fa4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e136cadea2d438b81f7a8ab29d47b05', 'request_id': 'req-f69d34cf-05ad-4662-ad87-02bce894fa4e'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:46.579501 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-f69d34cf-05ad-4662-ad87-02bce894fa4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e136cadea2d438b81f7a8ab29d47b05', 'request_id': 'req-f69d34cf-05ad-4662-ad87-02bce894fa4e'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:46.585860 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-f69d34cf-05ad-4662-ad87-02bce894fa4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e136cadea2d438b81f7a8ab29d47b05', 'request_id': 'req-f69d34cf-05ad-4662-ad87-02bce894fa4e'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:46.705855 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-976714a7-f405-4ca9-bb4d-87f6ef25a4ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2818f02432344270b2ae3c1212432e01', 'request_id': 'req-976714a7-f405-4ca9-bb4d-87f6ef25a4ef'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:46.711237 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-976714a7-f405-4ca9-bb4d-87f6ef25a4ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2818f02432344270b2ae3c1212432e01', 'request_id': 'req-976714a7-f405-4ca9-bb4d-87f6ef25a4ef'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:46.717760 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-976714a7-f405-4ca9-bb4d-87f6ef25a4ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2818f02432344270b2ae3c1212432e01', 'request_id': 'req-976714a7-f405-4ca9-bb4d-87f6ef25a4ef'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:46.723454 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-f69d34cf-05ad-4662-ad87-02bce894fa4e admin admin] The token cache is being invalidate because project 1e136cadea2d438b81f7a8ab29d47b05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:41:46.727851 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-f69d34cf-05ad-4662-ad87-02bce894fa4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:46.727851 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-f69d34cf-05ad-4662-ad87-02bce894fa4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:46.727851 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1376/2777] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:41:46 2022] DELETE /identity/v3/projects/1e136cadea2d438b81f7a8ab29d47b05 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:46.727851 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d34cf-05ad-4662-ad87-02bce894fa4e admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:46.729153 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-f69d34cf-05ad-4662-ad87-02bce894fa4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:46.796141 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d34cf-05ad-4662-ad87-02bce894fa4e admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:46.848735 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-976714a7-f405-4ca9-bb4d-87f6ef25a4ef admin admin] The token cache is being invalidate because project 2818f02432344270b2ae3c1212432e01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:41:46.849035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-976714a7-f405-4ca9-bb4d-87f6ef25a4ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:46.849680 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-976714a7-f405-4ca9-bb4d-87f6ef25a4ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:46.851791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1402/2778] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:41:46 2022] DELETE /identity/v3/projects/2818f02432344270b2ae3c1212432e01 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:46.852586 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976714a7-f405-4ca9-bb4d-87f6ef25a4ef admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:46.853708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca24741b-5e39-4497-aa5d-259bfc8d68fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:46.854306 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24741b-5e39-4497-aa5d-259bfc8d68fc admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:46.854597 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24741b-5e39-4497-aa5d-259bfc8d68fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:46.854876 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24741b-5e39-4497-aa5d-259bfc8d68fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:46.855744 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca24741b-5e39-4497-aa5d-259bfc8d68fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:46.855934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-976714a7-f405-4ca9-bb4d-87f6ef25a4ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:46.857826 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca24741b-5e39-4497-aa5d-259bfc8d68fc admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:46.869868 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976714a7-f405-4ca9-bb4d-87f6ef25a4ef admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:46.875345 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dd0119-6f07-4326-9c1e-7fa42c495a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:46.876257 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd0119-6f07-4326-9c1e-7fa42c495a1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:46.876536 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd0119-6f07-4326-9c1e-7fa42c495a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:46.876874 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd0119-6f07-4326-9c1e-7fa42c495a1a admin admin] PATH_INFO: `/v3/users/6162eb6bbee34972aa93e508688ee8fc` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:46.878167 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dd0119-6f07-4326-9c1e-7fa42c495a1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6162eb6bbee34972aa93e508688ee8fc)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:46.880149 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dd0119-6f07-4326-9c1e-7fa42c495a1a admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:46.945966 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1377/2779] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:41:46 2022] POST /identity/v3/projects => generated 385 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:41:46.947335 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d42c0b-92c1-45c6-ba86-108bf61c1cbf None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:46.947680 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d42c0b-92c1-45c6-ba86-108bf61c1cbf None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:46.947966 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d42c0b-92c1-45c6-ba86-108bf61c1cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:46.980049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:46.982241 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-52d42c0b-92c1-45c6-ba86-108bf61c1cbf None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:46.983720 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-52d42c0b-92c1-45c6-ba86-108bf61c1cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:47.031494 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-98dd0119-6f07-4326-9c1e-7fa42c495a1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6162eb6bbee34972aa93e508688ee8fc'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:47.032889 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-98dd0119-6f07-4326-9c1e-7fa42c495a1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6162eb6bbee34972aa93e508688ee8fc'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:47.040958 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-98dd0119-6f07-4326-9c1e-7fa42c495a1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6162eb6bbee34972aa93e508688ee8fc'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:47.044819 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-98dd0119-6f07-4326-9c1e-7fa42c495a1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6162eb6bbee34972aa93e508688ee8fc'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:47.046999 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1378/2780] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:47.048160 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d42c0b-92c1-45c6-ba86-108bf61c1cbf None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:47.049254 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-52d42c0b-92c1-45c6-ba86-108bf61c1cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:47.061258 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-98dd0119-6f07-4326-9c1e-7fa42c495a1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6162eb6bbee34972aa93e508688ee8fc'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:47.063164 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d42c0b-92c1-45c6-ba86-108bf61c1cbf None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:47.067837 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0741b-7180-4be7-a069-cfc4189b2e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:47.068618 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0741b-7180-4be7-a069-cfc4189b2e19 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:47.068979 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0741b-7180-4be7-a069-cfc4189b2e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:47.069332 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0741b-7180-4be7-a069-cfc4189b2e19 admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:47.070167 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c0741b-7180-4be7-a069-cfc4189b2e19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:47.072250 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c0741b-7180-4be7-a069-cfc4189b2e19 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:47.094988 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:47.135298 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1403/2781] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:41:46 2022] DELETE /identity/v3/users/6162eb6bbee34972aa93e508688ee8fc => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:47.155108 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175900c2-a8b8-4ff6-a3ee-a21a3c6b33b3 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:47.155108 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175900c2-a8b8-4ff6-a3ee-a21a3c6b33b3 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:47.155108 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175900c2-a8b8-4ff6-a3ee-a21a3c6b33b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:47.164226 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-175900c2-a8b8-4ff6-a3ee-a21a3c6b33b3 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:47.165621 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-175900c2-a8b8-4ff6-a3ee-a21a3c6b33b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:47.183738 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1379/2782] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:41:47 2022] POST /identity/v3/users => generated 299 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:41:47.186432 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0741b-7180-4be7-a069-cfc4189b2e19 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:47.197688 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0741b-7180-4be7-a069-cfc4189b2e19 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:47.231992 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1404/2783] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:47.236597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175900c2-a8b8-4ff6-a3ee-a21a3c6b33b3 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:47.252457 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175900c2-a8b8-4ff6-a3ee-a21a3c6b33b3 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:47.258981 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d9ab00-27fc-4776-b8d8-563fc60a19e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:47.260102 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9ab00-27fc-4776-b8d8-563fc60a19e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:47.260391 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9ab00-27fc-4776-b8d8-563fc60a19e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:47.260671 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9ab00-27fc-4776-b8d8-563fc60a19e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:47.261475 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d9ab00-27fc-4776-b8d8-563fc60a19e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:41:47.261712 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d9ab00-27fc-4776-b8d8-563fc60a19e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:47.263305 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d9ab00-27fc-4776-b8d8-563fc60a19e6 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:47.279710 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1405/2784] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:41:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:41:47.282647 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d9ab00-27fc-4776-b8d8-563fc60a19e6 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:47.294229 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d9ab00-27fc-4776-b8d8-563fc60a19e6 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:47.300938 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538948ab-1770-49ba-ace8-f9ff2121466b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:47.301821 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538948ab-1770-49ba-ace8-f9ff2121466b admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:47.302219 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538948ab-1770-49ba-ace8-f9ff2121466b admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:47.302488 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10002387-f276-4eb0-9613-3300e173106d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:47.302674 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538948ab-1770-49ba-ace8-f9ff2121466b admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:47.303360 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10002387-f276-4eb0-9613-3300e173106d admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:47.303541 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538948ab-1770-49ba-ace8-f9ff2121466b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:47.303685 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10002387-f276-4eb0-9613-3300e173106d admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:47.303948 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10002387-f276-4eb0-9613-3300e173106d admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:47.304721 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10002387-f276-4eb0-9613-3300e173106d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:47.306004 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538948ab-1770-49ba-ace8-f9ff2121466b admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:47.306304 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10002387-f276-4eb0-9613-3300e173106d admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:47.320397 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1380/2785] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:47 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 15:41:47.327264 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10002387-f276-4eb0-9613-3300e173106d admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:47.341238 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10002387-f276-4eb0-9613-3300e173106d admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:47.347878 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df489c44-ea19-432d-b2e6-67d4525eb76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:47.349309 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df489c44-ea19-432d-b2e6-67d4525eb76c admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:47.349991 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df489c44-ea19-432d-b2e6-67d4525eb76c admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:47.350427 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df489c44-ea19-432d-b2e6-67d4525eb76c admin admin] PATH_INFO: `/v3/projects/0d68f8aa14cf4e96b936c100efc198f2/users/3c2aa80580734f5ebeaa96e7a2e39a0b/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:47.374148 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df489c44-ea19-432d-b2e6-67d4525eb76c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d68f8aa14cf4e96b936c100efc198f2, user_id=3c2aa80580734f5ebeaa96e7a2e39a0b, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:47.377479 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df489c44-ea19-432d-b2e6-67d4525eb76c admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:47.432359 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1406/2786] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:41:47 2022] POST /identity/v3/projects => generated 385 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:41:47.436322 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538948ab-1770-49ba-ace8-f9ff2121466b admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:47.456678 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1381/2787] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:41:47 2022] PUT /identity/v3/projects/0d68f8aa14cf4e96b936c100efc198f2/users/3c2aa80580734f5ebeaa96e7a2e39a0b/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:41:47.462671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df489c44-ea19-432d-b2e6-67d4525eb76c admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:47.540989 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:47.544394 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538948ab-1770-49ba-ace8-f9ff2121466b admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:47.557930 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df489c44-ea19-432d-b2e6-67d4525eb76c admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:47.563278 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c56df-135a-42ad-b78a-1440b226fe06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:47.563976 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c56df-135a-42ad-b78a-1440b226fe06 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:47.564263 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c56df-135a-42ad-b78a-1440b226fe06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:47.564540 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c56df-135a-42ad-b78a-1440b226fe06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:47.565269 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68c56df-135a-42ad-b78a-1440b226fe06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:47.566900 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68c56df-135a-42ad-b78a-1440b226fe06 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:47.577638 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1382/2788] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:47 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:41:47.581638 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c56df-135a-42ad-b78a-1440b226fe06 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:47.593675 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c56df-135a-42ad-b78a-1440b226fe06 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:47.612454 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9056ca-a952-41d5-9a5a-0dc7b9b0de81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:47.614831 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9056ca-a952-41d5-9a5a-0dc7b9b0de81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:47.614831 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9056ca-a952-41d5-9a5a-0dc7b9b0de81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:47.615749 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9056ca-a952-41d5-9a5a-0dc7b9b0de81 admin admin] PATH_INFO: `/v3/projects/0d68f8aa14cf4e96b936c100efc198f2/users/3c2aa80580734f5ebeaa96e7a2e39a0b/roles/0e7793fbf3064037984da651879e3a94` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:47.621028 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9056ca-a952-41d5-9a5a-0dc7b9b0de81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d68f8aa14cf4e96b936c100efc198f2, user_id=3c2aa80580734f5ebeaa96e7a2e39a0b, role_id=0e7793fbf3064037984da651879e3a94)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:47.622825 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3742af4a-7a19-429f-ba4e-63a568282e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:47.625241 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3742af4a-7a19-429f-ba4e-63a568282e67 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:47.625609 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3742af4a-7a19-429f-ba4e-63a568282e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:47.627101 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9056ca-a952-41d5-9a5a-0dc7b9b0de81 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:47.627465 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3742af4a-7a19-429f-ba4e-63a568282e67 admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:47.628270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3742af4a-7a19-429f-ba4e-63a568282e67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:47.630257 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3742af4a-7a19-429f-ba4e-63a568282e67 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:47.652242 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1383/2789] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:41:47 2022] PUT /identity/v3/projects/0d68f8aa14cf4e96b936c100efc198f2/users/3c2aa80580734f5ebeaa96e7a2e39a0b/roles/0e7793fbf3064037984da651879e3a94 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:41:47.652242 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9056ca-a952-41d5-9a5a-0dc7b9b0de81 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:47.655712 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-9b9056ca-a952-41d5-9a5a-0dc7b9b0de81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:47.689714 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:47.719756 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1407/2790] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:41:47 2022] POST /identity/v3/users => generated 293 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:41:47.722607 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3742af4a-7a19-429f-ba4e-63a568282e67 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:47.749736 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3742af4a-7a19-429f-ba4e-63a568282e67 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:47.768450 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9056ca-a952-41d5-9a5a-0dc7b9b0de81 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:47.774297 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0073fc9a-2b12-4bcb-8cab-601ccdffaa80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:47.775271 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073fc9a-2b12-4bcb-8cab-601ccdffaa80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:47.775651 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073fc9a-2b12-4bcb-8cab-601ccdffaa80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:47.775995 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073fc9a-2b12-4bcb-8cab-601ccdffaa80 admin admin] PATH_INFO: `/v3/projects/449e15f8dbad4fdb90b2f2dfc76e7889` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:47.777424 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0073fc9a-2b12-4bcb-8cab-601ccdffaa80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=449e15f8dbad4fdb90b2f2dfc76e7889)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:47.779513 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0073fc9a-2b12-4bcb-8cab-601ccdffaa80 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:47.831373 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aaf9b5-55df-4cc3-9e80-9236c40b142f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:47.832176 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aaf9b5-55df-4cc3-9e80-9236c40b142f admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:47.832848 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aaf9b5-55df-4cc3-9e80-9236c40b142f admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:47.832848 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aaf9b5-55df-4cc3-9e80-9236c40b142f admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:47.833701 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aaf9b5-55df-4cc3-9e80-9236c40b142f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:47.835755 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aaf9b5-55df-4cc3-9e80-9236c40b142f admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:47.849594 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1408/2791] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:47 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:41:47.849594 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aaf9b5-55df-4cc3-9e80-9236c40b142f admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:47.849594 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-09aaf9b5-55df-4cc3-9e80-9236c40b142f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:47.906307 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-0073fc9a-2b12-4bcb-8cab-601ccdffaa80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '449e15f8dbad4fdb90b2f2dfc76e7889', 'request_id': 'req-0073fc9a-2b12-4bcb-8cab-601ccdffaa80'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:47.907934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aaf9b5-55df-4cc3-9e80-9236c40b142f admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:47.912007 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-0073fc9a-2b12-4bcb-8cab-601ccdffaa80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '449e15f8dbad4fdb90b2f2dfc76e7889', 'request_id': 'req-0073fc9a-2b12-4bcb-8cab-601ccdffaa80'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:47.919846 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-0073fc9a-2b12-4bcb-8cab-601ccdffaa80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '449e15f8dbad4fdb90b2f2dfc76e7889', 'request_id': 'req-0073fc9a-2b12-4bcb-8cab-601ccdffaa80'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:47.959680 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b7ecf-3bb3-4d37-8c6d-f4354390c9ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:47.960347 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b7ecf-3bb3-4d37-8c6d-f4354390c9ca service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:47.960702 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b7ecf-3bb3-4d37-8c6d-f4354390c9ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:47.960971 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b7ecf-3bb3-4d37-8c6d-f4354390c9ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:47.962985 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-c29b7ecf-3bb3-4d37-8c6d-f4354390c9ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:48.015501 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-0073fc9a-2b12-4bcb-8cab-601ccdffaa80 admin admin] The token cache is being invalidate because project 449e15f8dbad4fdb90b2f2dfc76e7889 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:41:48.020034 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-0073fc9a-2b12-4bcb-8cab-601ccdffaa80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:48.020034 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-0073fc9a-2b12-4bcb-8cab-601ccdffaa80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:48.020034 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1384/2792] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:41:47 2022] DELETE /identity/v3/projects/449e15f8dbad4fdb90b2f2dfc76e7889 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:48.020034 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0073fc9a-2b12-4bcb-8cab-601ccdffaa80 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:48.021582 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-0073fc9a-2b12-4bcb-8cab-601ccdffaa80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:48.064200 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29b7ecf-3bb3-4d37-8c6d-f4354390c9ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:48.065870 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29b7ecf-3bb3-4d37-8c6d-f4354390c9ca service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:48.083998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1409/2793] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:41:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:41:48.083998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af219f-f2d2-4390-8190-601143f55d9b None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:48.083998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af219f-f2d2-4390-8190-601143f55d9b None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:48.083998 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af219f-f2d2-4390-8190-601143f55d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:48.106734 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-08af219f-f2d2-4390-8190-601143f55d9b None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:48.108836 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-08af219f-f2d2-4390-8190-601143f55d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:48.124246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1410/2794] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:48.128326 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8995ec0-117c-435f-bfd0-e53907a74ac3 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:48.128806 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8995ec0-117c-435f-bfd0-e53907a74ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:48.129373 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8995ec0-117c-435f-bfd0-e53907a74ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:48.149732 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0073fc9a-2b12-4bcb-8cab-601ccdffaa80 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:48.154484 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-e8995ec0-117c-435f-bfd0-e53907a74ac3 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:48.156902 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a434f0-ca34-4c68-a119-d8eda5ee1fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:48.157663 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a434f0-ca34-4c68-a119-d8eda5ee1fe2 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:48.157791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-e8995ec0-117c-435f-bfd0-e53907a74ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:48.157946 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a434f0-ca34-4c68-a119-d8eda5ee1fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:48.158195 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a434f0-ca34-4c68-a119-d8eda5ee1fe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:48.158972 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a434f0-ca34-4c68-a119-d8eda5ee1fe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:48.162101 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a434f0-ca34-4c68-a119-d8eda5ee1fe2 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:48.169641 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1411/2795] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:48.181043 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:48.182495 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8995ec0-117c-435f-bfd0-e53907a74ac3 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:48.186759 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1385/2796] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:48 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:41:48.190879 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a434f0-ca34-4c68-a119-d8eda5ee1fe2 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:48.208234 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a434f0-ca34-4c68-a119-d8eda5ee1fe2 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:48.208617 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8995ec0-117c-435f-bfd0-e53907a74ac3 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:48.221279 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3527cb-54bc-45c0-8097-714f85f3c90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:48.222029 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e48f466-1811-4c75-9b5a-e05d427cf672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:48.222934 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e48f466-1811-4c75-9b5a-e05d427cf672 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:48.223213 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e48f466-1811-4c75-9b5a-e05d427cf672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:48.223419 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3527cb-54bc-45c0-8097-714f85f3c90d admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:48.223774 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e48f466-1811-4c75-9b5a-e05d427cf672 admin admin] PATH_INFO: `/v3/projects/a2a7a805b01d4f81923b0382140f1c25/users/ad2d070ff0344ca89d7d4a565ffdc1dc/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:48.223905 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3527cb-54bc-45c0-8097-714f85f3c90d admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:48.224762 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3527cb-54bc-45c0-8097-714f85f3c90d admin admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:48.226963 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3527cb-54bc-45c0-8097-714f85f3c90d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:41:48.227555 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3527cb-54bc-45c0-8097-714f85f3c90d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:48.229597 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3527cb-54bc-45c0-8097-714f85f3c90d admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:48.247617 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1412/2797] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:41:48.249757 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3527cb-54bc-45c0-8097-714f85f3c90d admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:48.256245 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e48f466-1811-4c75-9b5a-e05d427cf672 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2a7a805b01d4f81923b0382140f1c25, user_id=ad2d070ff0344ca89d7d4a565ffdc1dc, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:48.260616 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e48f466-1811-4c75-9b5a-e05d427cf672 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:48.264778 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3527cb-54bc-45c0-8097-714f85f3c90d admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:48.273201 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413465b1-52a7-4f46-ac80-3af2751c14f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:48.274386 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413465b1-52a7-4f46-ac80-3af2751c14f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:48.274715 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413465b1-52a7-4f46-ac80-3af2751c14f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:48.275004 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413465b1-52a7-4f46-ac80-3af2751c14f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:48.275741 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413465b1-52a7-4f46-ac80-3af2751c14f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:48.277897 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413465b1-52a7-4f46-ac80-3af2751c14f9 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:48.376294 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1386/2798] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:41:48 2022] PUT /identity/v3/projects/a2a7a805b01d4f81923b0382140f1c25/users/ad2d070ff0344ca89d7d4a565ffdc1dc/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:41:48.389629 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5b961-2782-49cf-ac74-d6b19319aa73 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:48.390412 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5b961-2782-49cf-ac74-d6b19319aa73 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:48.390763 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5b961-2782-49cf-ac74-d6b19319aa73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:48.419246 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1413/2799] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:41:48 2022] POST /identity/v3/projects => generated 371 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:41:48.423434 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413465b1-52a7-4f46-ac80-3af2751c14f9 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:48.450227 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-9ac5b961-2782-49cf-ac74-d6b19319aa73 None None] MFA Rules not processed for user `ad2d070ff0344ca89d7d4a565ffdc1dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:48.452883 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-9ac5b961-2782-49cf-ac74-d6b19319aa73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:48.578864 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413465b1-52a7-4f46-ac80-3af2751c14f9 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:48.590346 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2954d5d-111e-4e46-a3e9-645997ea5804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:48.591044 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2954d5d-111e-4e46-a3e9-645997ea5804 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:48.591330 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2954d5d-111e-4e46-a3e9-645997ea5804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:48.591624 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2954d5d-111e-4e46-a3e9-645997ea5804 admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:48.592323 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2954d5d-111e-4e46-a3e9-645997ea5804 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:48.597419 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2954d5d-111e-4e46-a3e9-645997ea5804 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:48.609515 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1387/2800] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:48.618093 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5b961-2782-49cf-ac74-d6b19319aa73 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:48.619608 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-9ac5b961-2782-49cf-ac74-d6b19319aa73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:48.715222 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5b961-2782-49cf-ac74-d6b19319aa73 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:48.719433 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a5710a-c97e-4576-8647-1476e1770da5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:48.720492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5710a-c97e-4576-8647-1476e1770da5 service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:48.721254 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5710a-c97e-4576-8647-1476e1770da5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:48.721775 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5710a-c97e-4576-8647-1476e1770da5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:48.734005 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a5710a-c97e-4576-8647-1476e1770da5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:48.736147 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a5710a-c97e-4576-8647-1476e1770da5 service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:48.749331 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1388/2801] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:41:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 15:41:48.753613 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1414/2802] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:41:48 2022] POST /identity/v3/users => generated 286 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:41:48.758410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a5710a-c97e-4576-8647-1476e1770da5 service cinder] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:48.775671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a5710a-c97e-4576-8647-1476e1770da5 service cinder] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:48.781135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f5e606-c689-43f3-9dfc-9979cc3a9569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:48.781806 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5e606-c689-43f3-9dfc-9979cc3a9569 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:48.782068 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5e606-c689-43f3-9dfc-9979cc3a9569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:48.782352 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5e606-c689-43f3-9dfc-9979cc3a9569 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:48.783232 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f5e606-c689-43f3-9dfc-9979cc3a9569 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:48.784947 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f5e606-c689-43f3-9dfc-9979cc3a9569 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:48.802648 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1389/2803] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:48 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:41:48.804563 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2954d5d-111e-4e46-a3e9-645997ea5804 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:48.820631 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2954d5d-111e-4e46-a3e9-645997ea5804 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:48.833358 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d1543c-2e10-4d8a-80e4-34a282ba3641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:48.838345 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d1543c-2e10-4d8a-80e4-34a282ba3641 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:48.838345 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d1543c-2e10-4d8a-80e4-34a282ba3641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:48.838345 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d1543c-2e10-4d8a-80e4-34a282ba3641 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:48.839068 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d1543c-2e10-4d8a-80e4-34a282ba3641 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:48.843453 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d1543c-2e10-4d8a-80e4-34a282ba3641 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:48.887038 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1415/2804] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:48 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:41:48.887038 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:48.888848 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f5e606-c689-43f3-9dfc-9979cc3a9569 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:48.904619 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f5e606-c689-43f3-9dfc-9979cc3a9569 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:48.919293 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d529c365-d1f1-47e1-b718-6345fa9f073a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:48.920904 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529c365-d1f1-47e1-b718-6345fa9f073a admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:48.921326 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529c365-d1f1-47e1-b718-6345fa9f073a admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:48.921605 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529c365-d1f1-47e1-b718-6345fa9f073a admin admin] PATH_INFO: `/v3/projects/40fbe382dfca42bcb4f8fc0c0d25fc04/users/fc3ca236686145749dc0733e13cb678e/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:48.968326 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d529c365-d1f1-47e1-b718-6345fa9f073a admin admin] RBAC: Authorizing `identity:create_grant(project_id=40fbe382dfca42bcb4f8fc0c0d25fc04, user_id=fc3ca236686145749dc0733e13cb678e, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:48.971474 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d529c365-d1f1-47e1-b718-6345fa9f073a admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:49.066077 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1390/2805] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:41:48 2022] PUT /identity/v3/projects/40fbe382dfca42bcb4f8fc0c0d25fc04/users/fc3ca236686145749dc0733e13cb678e/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:49.077245 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a984e8-e1c5-4f7b-b963-cb1fd0395320 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:49.080294 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a984e8-e1c5-4f7b-b963-cb1fd0395320 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:49.080294 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a984e8-e1c5-4f7b-b963-cb1fd0395320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:49.107413 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-b4a984e8-e1c5-4f7b-b963-cb1fd0395320 None None] MFA Rules not processed for user `fc3ca236686145749dc0733e13cb678e`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:49.108683 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-b4a984e8-e1c5-4f7b-b963-cb1fd0395320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:49.195723 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1416/2806] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:49.206297 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d529c365-d1f1-47e1-b718-6345fa9f073a admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:49.208661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-d529c365-d1f1-47e1-b718-6345fa9f073a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:49.292088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d529c365-d1f1-47e1-b718-6345fa9f073a admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:49.297328 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebcbf46-38f7-48b7-9442-1c2bd0e1d9ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:49.297499 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebcbf46-38f7-48b7-9442-1c2bd0e1d9ab service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:49.297763 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebcbf46-38f7-48b7-9442-1c2bd0e1d9ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:49.298036 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebcbf46-38f7-48b7-9442-1c2bd0e1d9ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:49.312221 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebcbf46-38f7-48b7-9442-1c2bd0e1d9ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:49.313850 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebcbf46-38f7-48b7-9442-1c2bd0e1d9ab service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:49.324815 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1391/2807] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:41:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:41:49.348492 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a984e8-e1c5-4f7b-b963-cb1fd0395320 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:49.457916 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:49.499557 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a984e8-e1c5-4f7b-b963-cb1fd0395320 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:49.510241 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dedfa9-3708-49f1-8f30-c215bbbca91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:49.511508 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dedfa9-3708-49f1-8f30-c215bbbca91d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:49.511823 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dedfa9-3708-49f1-8f30-c215bbbca91d admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:49.512101 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dedfa9-3708-49f1-8f30-c215bbbca91d admin admin] PATH_INFO: `/v3/users/fc3ca236686145749dc0733e13cb678e` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:49.514358 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dedfa9-3708-49f1-8f30-c215bbbca91d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc3ca236686145749dc0733e13cb678e)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:49.516365 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dedfa9-3708-49f1-8f30-c215bbbca91d admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:49.864255 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-c1dedfa9-3708-49f1-8f30-c215bbbca91d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc3ca236686145749dc0733e13cb678e'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:49.865973 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-c1dedfa9-3708-49f1-8f30-c215bbbca91d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc3ca236686145749dc0733e13cb678e'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:49.869976 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:49.879504 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-c1dedfa9-3708-49f1-8f30-c215bbbca91d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc3ca236686145749dc0733e13cb678e'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:49.880678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-c1dedfa9-3708-49f1-8f30-c215bbbca91d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc3ca236686145749dc0733e13cb678e'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:49.898208 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-c1dedfa9-3708-49f1-8f30-c215bbbca91d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc3ca236686145749dc0733e13cb678e'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:50.003752 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1417/2808] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:41:49 2022] DELETE /identity/v3/users/fc3ca236686145749dc0733e13cb678e => generated 0 bytes in 652 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:50.005160 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3008c3-f4de-4784-b7e6-2f8d9940d8ef None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:50.005457 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3008c3-f4de-4784-b7e6-2f8d9940d8ef None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:50.005738 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3008c3-f4de-4784-b7e6-2f8d9940d8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:50.036001 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-fe3008c3-f4de-4784-b7e6-2f8d9940d8ef None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:50.037449 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-fe3008c3-f4de-4784-b7e6-2f8d9940d8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:50.129471 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1392/2809] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:41:50.144011 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dedfa9-3708-49f1-8f30-c215bbbca91d admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:50.146329 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-c1dedfa9-3708-49f1-8f30-c215bbbca91d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:50.245987 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dedfa9-3708-49f1-8f30-c215bbbca91d admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:50.249885 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503a1a9e-d964-45f7-ac95-df9bebbb0dac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:50.250550 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a1a9e-d964-45f7-ac95-df9bebbb0dac service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:50.250832 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a1a9e-d964-45f7-ac95-df9bebbb0dac service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:50.251109 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a1a9e-d964-45f7-ac95-df9bebbb0dac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:50.264067 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503a1a9e-d964-45f7-ac95-df9bebbb0dac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:50.265566 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503a1a9e-d964-45f7-ac95-df9bebbb0dac service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:50.278903 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1418/2810] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:41:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:41:50.949329 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3008c3-f4de-4784-b7e6-2f8d9940d8ef None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:50.952338 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-fe3008c3-f4de-4784-b7e6-2f8d9940d8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:50.983274 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3008c3-f4de-4784-b7e6-2f8d9940d8ef None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:50.990560 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499d231-5f9a-4a93-8a9a-7ee6e8945f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:50.992774 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499d231-5f9a-4a93-8a9a-7ee6e8945f06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:50.993306 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499d231-5f9a-4a93-8a9a-7ee6e8945f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:50.993812 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499d231-5f9a-4a93-8a9a-7ee6e8945f06 admin admin] PATH_INFO: `/v3/projects/40fbe382dfca42bcb4f8fc0c0d25fc04` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:50.995390 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5499d231-5f9a-4a93-8a9a-7ee6e8945f06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40fbe382dfca42bcb4f8fc0c0d25fc04)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:50.997754 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5499d231-5f9a-4a93-8a9a-7ee6e8945f06 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:51.171423 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-5499d231-5f9a-4a93-8a9a-7ee6e8945f06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40fbe382dfca42bcb4f8fc0c0d25fc04', 'request_id': 'req-5499d231-5f9a-4a93-8a9a-7ee6e8945f06'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:51.183689 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-5499d231-5f9a-4a93-8a9a-7ee6e8945f06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40fbe382dfca42bcb4f8fc0c0d25fc04', 'request_id': 'req-5499d231-5f9a-4a93-8a9a-7ee6e8945f06'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:51.191507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-5499d231-5f9a-4a93-8a9a-7ee6e8945f06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40fbe382dfca42bcb4f8fc0c0d25fc04', 'request_id': 'req-5499d231-5f9a-4a93-8a9a-7ee6e8945f06'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:51.294975 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-5499d231-5f9a-4a93-8a9a-7ee6e8945f06 admin admin] The token cache is being invalidate because project 40fbe382dfca42bcb4f8fc0c0d25fc04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:41:51.297419 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-5499d231-5f9a-4a93-8a9a-7ee6e8945f06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:51.297862 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-5499d231-5f9a-4a93-8a9a-7ee6e8945f06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:51.300251 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1393/2811] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:41:50 2022] DELETE /identity/v3/projects/40fbe382dfca42bcb4f8fc0c0d25fc04 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:51.305683 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf43438-e1c5-478d-8aca-5354d6dc29d4 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:51.306199 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf43438-e1c5-478d-8aca-5354d6dc29d4 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:51.306307 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf43438-e1c5-478d-8aca-5354d6dc29d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:51.328577 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-1cf43438-e1c5-478d-8aca-5354d6dc29d4 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:51.330509 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-1cf43438-e1c5-478d-8aca-5354d6dc29d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:51.405103 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:INSERT=2,keystone:SELECT=89,keystone:DELETE=9 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:51.424131 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1419/2812] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:51.436586 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3fc8f-cb43-4ccd-917a-640a809c1334 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:51.436586 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3fc8f-cb43-4ccd-917a-640a809c1334 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:51.436586 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3fc8f-cb43-4ccd-917a-640a809c1334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:51.460307 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-98b3fc8f-cb43-4ccd-917a-640a809c1334 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:51.462129 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-98b3fc8f-cb43-4ccd-917a-640a809c1334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:51.474506 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1394/2813] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:51.485458 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf43438-e1c5-478d-8aca-5354d6dc29d4 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:51.501055 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf43438-e1c5-478d-8aca-5354d6dc29d4 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:51.509496 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca09f66f-0947-4a34-a906-5e0505379a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:51.510624 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca09f66f-0947-4a34-a906-5e0505379a59 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:51.511158 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca09f66f-0947-4a34-a906-5e0505379a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:51.511618 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca09f66f-0947-4a34-a906-5e0505379a59 admin admin] PATH_INFO: `/v3/domains` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:51.513002 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca09f66f-0947-4a34-a906-5e0505379a59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80301) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:41:51.513541 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca09f66f-0947-4a34-a906-5e0505379a59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:51.524222 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca09f66f-0947-4a34-a906-5e0505379a59 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:51.550217 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1420/2814] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:41:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:41:51.554652 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b3fc8f-cb43-4ccd-917a-640a809c1334 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:51.564143 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b3fc8f-cb43-4ccd-917a-640a809c1334 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:51.572708 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0348e0-29fd-4fcc-a974-61127c79b759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:51.573413 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0348e0-29fd-4fcc-a974-61127c79b759 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:51.573684 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0348e0-29fd-4fcc-a974-61127c79b759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:51.574420 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0348e0-29fd-4fcc-a974-61127c79b759 admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:51.574629 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0348e0-29fd-4fcc-a974-61127c79b759 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:51.576665 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0348e0-29fd-4fcc-a974-61127c79b759 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:51.594822 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:51.692276 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1395/2815] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:41:51 2022] POST /identity/v3/projects => generated 371 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:41:51.696730 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca09f66f-0947-4a34-a906-5e0505379a59 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:51.707336 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca09f66f-0947-4a34-a906-5e0505379a59 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:51.791587 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e04b700-2587-4609-bd60-fd6e28deefd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:51.792307 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04b700-2587-4609-bd60-fd6e28deefd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:51.792682 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04b700-2587-4609-bd60-fd6e28deefd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:51.793054 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04b700-2587-4609-bd60-fd6e28deefd8 admin admin] PATH_INFO: `/v3/users` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:51.793843 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e04b700-2587-4609-bd60-fd6e28deefd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:51.796023 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e04b700-2587-4609-bd60-fd6e28deefd8 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:51.908540 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1421/2816] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:41:51 2022] POST /identity/v3/users => generated 286 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:41:51.912836 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0348e0-29fd-4fcc-a974-61127c79b759 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:51.924545 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0348e0-29fd-4fcc-a974-61127c79b759 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:51.931216 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d2384-7cf7-45c1-b731-62ad068396a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:51.931867 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d2384-7cf7-45c1-b731-62ad068396a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:51.932281 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d2384-7cf7-45c1-b731-62ad068396a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:51.932688 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d2384-7cf7-45c1-b731-62ad068396a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:51.933614 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d2384-7cf7-45c1-b731-62ad068396a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:51.935960 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d2384-7cf7-45c1-b731-62ad068396a1 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:51.954243 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1396/2817] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:41:51.958268 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e04b700-2587-4609-bd60-fd6e28deefd8 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:51.971818 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e04b700-2587-4609-bd60-fd6e28deefd8 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:51.977693 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb6968-ca27-48d9-b89a-08b65ce80ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:51.978550 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb6968-ca27-48d9-b89a-08b65ce80ffa admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:51.978913 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb6968-ca27-48d9-b89a-08b65ce80ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:51.979265 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb6968-ca27-48d9-b89a-08b65ce80ffa admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:51.980155 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbb6968-ca27-48d9-b89a-08b65ce80ffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:51.981878 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbb6968-ca27-48d9-b89a-08b65ce80ffa admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:51.994280 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1422/2818] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:41:52.001753 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d2384-7cf7-45c1-b731-62ad068396a1 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:52.017994 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d2384-7cf7-45c1-b731-62ad068396a1 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:52.032058 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7078f5c9-9735-436e-a388-ca587068a7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:52.032058 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078f5c9-9735-436e-a388-ca587068a7f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:52.032058 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078f5c9-9735-436e-a388-ca587068a7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:52.032058 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078f5c9-9735-436e-a388-ca587068a7f0 admin admin] PATH_INFO: `/v3/projects/f90b44820e424210b4dea0abc23ea469/users/03f8ed4d3d5a4447ba2c61b3dd4c314a/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:52.049038 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7078f5c9-9735-436e-a388-ca587068a7f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f90b44820e424210b4dea0abc23ea469, user_id=03f8ed4d3d5a4447ba2c61b3dd4c314a, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:52.051819 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7078f5c9-9735-436e-a388-ca587068a7f0 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:52.151003 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1397/2819] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:41:51 2022] PUT /identity/v3/projects/f90b44820e424210b4dea0abc23ea469/users/03f8ed4d3d5a4447ba2c61b3dd4c314a/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:52.155965 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc2123c-3a1d-4726-84eb-38436c69be3c None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:52.156556 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc2123c-3a1d-4726-84eb-38436c69be3c None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:52.157035 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc2123c-3a1d-4726-84eb-38436c69be3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:52.179531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-0cc2123c-3a1d-4726-84eb-38436c69be3c None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:52.180791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-0cc2123c-3a1d-4726-84eb-38436c69be3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:52.256560 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1423/2820] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 15:41:52.269482 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7078f5c9-9735-436e-a388-ca587068a7f0 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:52.271387 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-7078f5c9-9735-436e-a388-ca587068a7f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:52.328378 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc2123c-3a1d-4726-84eb-38436c69be3c None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:52.330810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-0cc2123c-3a1d-4726-84eb-38436c69be3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:52.349009 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc2123c-3a1d-4726-84eb-38436c69be3c None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:52.350224 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7078f5c9-9735-436e-a388-ca587068a7f0 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:52.354198 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171f8ac-4fda-46b9-a9d9-c70065a9121a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:52.354845 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171f8ac-4fda-46b9-a9d9-c70065a9121a service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:52.355211 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171f8ac-4fda-46b9-a9d9-c70065a9121a service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:52.355326 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0faf9d-75b0-497c-87bc-d2d14530745f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:52.355452 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171f8ac-4fda-46b9-a9d9-c70065a9121a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:52.356168 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0faf9d-75b0-497c-87bc-d2d14530745f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:52.356555 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0faf9d-75b0-497c-87bc-d2d14530745f admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:52.356832 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0faf9d-75b0-497c-87bc-d2d14530745f admin admin] PATH_INFO: `/v3/users/ad2d070ff0344ca89d7d4a565ffdc1dc` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:52.358362 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0faf9d-75b0-497c-87bc-d2d14530745f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad2d070ff0344ca89d7d4a565ffdc1dc)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:52.360622 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0faf9d-75b0-497c-87bc-d2d14530745f admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:52.367494 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a171f8ac-4fda-46b9-a9d9-c70065a9121a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:52.369123 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a171f8ac-4fda-46b9-a9d9-c70065a9121a service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:52.381635 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1398/2821] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:41:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:41:52.625689 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-2e0faf9d-75b0-497c-87bc-d2d14530745f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad2d070ff0344ca89d7d4a565ffdc1dc'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:52.628080 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-2e0faf9d-75b0-497c-87bc-d2d14530745f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad2d070ff0344ca89d7d4a565ffdc1dc'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:52.637150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-2e0faf9d-75b0-497c-87bc-d2d14530745f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad2d070ff0344ca89d7d4a565ffdc1dc'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:52.640947 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-2e0faf9d-75b0-497c-87bc-d2d14530745f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad2d070ff0344ca89d7d4a565ffdc1dc'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:52.654017 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-2e0faf9d-75b0-497c-87bc-d2d14530745f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad2d070ff0344ca89d7d4a565ffdc1dc'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:52.813033 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1424/2822] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:41:52 2022] DELETE /identity/v3/users/ad2d070ff0344ca89d7d4a565ffdc1dc => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:52.813249 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd5170-8f4e-4387-9e12-6d5c5f5e6760 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:52.813521 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd5170-8f4e-4387-9e12-6d5c5f5e6760 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:52.813803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd5170-8f4e-4387-9e12-6d5c5f5e6760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:52.832304 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:52.833733 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-a5dd5170-8f4e-4387-9e12-6d5c5f5e6760 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:52.835712 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-a5dd5170-8f4e-4387-9e12-6d5c5f5e6760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:52.903866 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1399/2823] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:52.943732 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0faf9d-75b0-497c-87bc-d2d14530745f admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:52.945813 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-2e0faf9d-75b0-497c-87bc-d2d14530745f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:53.059681 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0faf9d-75b0-497c-87bc-d2d14530745f admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:53.066552 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75756648-0dd7-40dc-9d77-b9cf41ccfb19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:53.067321 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75756648-0dd7-40dc-9d77-b9cf41ccfb19 service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:53.067785 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75756648-0dd7-40dc-9d77-b9cf41ccfb19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:53.068166 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75756648-0dd7-40dc-9d77-b9cf41ccfb19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:53.081337 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75756648-0dd7-40dc-9d77-b9cf41ccfb19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:53.083215 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75756648-0dd7-40dc-9d77-b9cf41ccfb19 service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:53.097586 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1425/2824] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:41:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:41:53.818846 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dd5170-8f4e-4387-9e12-6d5c5f5e6760 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:53.823649 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-a5dd5170-8f4e-4387-9e12-6d5c5f5e6760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:53.870642 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dd5170-8f4e-4387-9e12-6d5c5f5e6760 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:53.896954 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e265729-a1d5-4f36-9e85-79f56735ef18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:53.898761 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e265729-a1d5-4f36-9e85-79f56735ef18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:53.899231 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e265729-a1d5-4f36-9e85-79f56735ef18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:53.899722 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e265729-a1d5-4f36-9e85-79f56735ef18 admin admin] PATH_INFO: `/v3/projects/a2a7a805b01d4f81923b0382140f1c25` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:53.903666 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e265729-a1d5-4f36-9e85-79f56735ef18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2a7a805b01d4f81923b0382140f1c25)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:53.908778 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e265729-a1d5-4f36-9e85-79f56735ef18 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:53.942771 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:54.033545 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-7e265729-a1d5-4f36-9e85-79f56735ef18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2a7a805b01d4f81923b0382140f1c25', 'request_id': 'req-7e265729-a1d5-4f36-9e85-79f56735ef18'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:54.041412 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-7e265729-a1d5-4f36-9e85-79f56735ef18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2a7a805b01d4f81923b0382140f1c25', 'request_id': 'req-7e265729-a1d5-4f36-9e85-79f56735ef18'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:54.050017 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-7e265729-a1d5-4f36-9e85-79f56735ef18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2a7a805b01d4f81923b0382140f1c25', 'request_id': 'req-7e265729-a1d5-4f36-9e85-79f56735ef18'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:54.158867 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-7e265729-a1d5-4f36-9e85-79f56735ef18 admin admin] The token cache is being invalidate because project a2a7a805b01d4f81923b0382140f1c25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:41:54.159595 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-7e265729-a1d5-4f36-9e85-79f56735ef18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:54.160896 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-7e265729-a1d5-4f36-9e85-79f56735ef18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:41:54.168709 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1400/2825] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:41:53 2022] DELETE /identity/v3/projects/a2a7a805b01d4f81923b0382140f1c25 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:41:54.175336 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128ab94-3f07-434d-923d-ef4140ddf69f None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:54.176298 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128ab94-3f07-434d-923d-ef4140ddf69f None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:54.176752 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128ab94-3f07-434d-923d-ef4140ddf69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:54.222997 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-0128ab94-3f07-434d-923d-ef4140ddf69f None None] MFA Rules not processed for user `03f8ed4d3d5a4447ba2c61b3dd4c314a`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:54.227295 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-0128ab94-3f07-434d-923d-ef4140ddf69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:54.320177 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:54.358051 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1426/2826] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:54.365274 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e265729-a1d5-4f36-9e85-79f56735ef18 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:54.368609 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-7e265729-a1d5-4f36-9e85-79f56735ef18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:54.498090 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e265729-a1d5-4f36-9e85-79f56735ef18 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:54.502200 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837b7436-8e67-43c1-b41f-97c361d899fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:54.502852 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b7436-8e67-43c1-b41f-97c361d899fe service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:54.503139 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b7436-8e67-43c1-b41f-97c361d899fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:54.503416 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b7436-8e67-43c1-b41f-97c361d899fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:54.513276 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837b7436-8e67-43c1-b41f-97c361d899fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:54.514713 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837b7436-8e67-43c1-b41f-97c361d899fe service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:54.536190 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1401/2827] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:41:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:41:55.746916 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5217f3-0aa0-40ca-a80a-597ea95f2011 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:55.747385 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5217f3-0aa0-40ca-a80a-597ea95f2011 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:55.747509 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5217f3-0aa0-40ca-a80a-597ea95f2011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:55.807828 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-2a5217f3-0aa0-40ca-a80a-597ea95f2011 None None] MFA Rules not processed for user `3c2aa80580734f5ebeaa96e7a2e39a0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:55.809131 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-2a5217f3-0aa0-40ca-a80a-597ea95f2011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:55.904268 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1427/2828] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:55 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:55.916531 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837b7436-8e67-43c1-b41f-97c361d899fe service cinder] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:55.918185 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-837b7436-8e67-43c1-b41f-97c361d899fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:56.002090 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837b7436-8e67-43c1-b41f-97c361d899fe service cinder] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:56.006320 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e7828-7320-43eb-82b3-e7d778ee9461 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0d29e718f446e2a9fd1b2575a86041', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:56.007505 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e7828-7320-43eb-82b3-e7d778ee9461 service glance] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:56.008019 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e7828-7320-43eb-82b3-e7d778ee9461 service glance] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:56.008305 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e7828-7320-43eb-82b3-e7d778ee9461 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:56.010473 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-846e7828-7320-43eb-82b3-e7d778ee9461 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:56.084613 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846e7828-7320-43eb-82b3-e7d778ee9461 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:56.088849 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846e7828-7320-43eb-82b3-e7d778ee9461 service glance] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:56.102869 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1402/2829] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:41:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:41:56.350023 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9becc3-72ec-4e85-9d6f-2124a05e1157 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:56.350558 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9becc3-72ec-4e85-9d6f-2124a05e1157 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:56.351080 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9becc3-72ec-4e85-9d6f-2124a05e1157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:56.370437 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-7c9becc3-72ec-4e85-9d6f-2124a05e1157 None None] MFA Rules not processed for user `cd3c04f39d774998b66e6ce920925691`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:56.371703 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-7c9becc3-72ec-4e85-9d6f-2124a05e1157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:56.465210 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1428/2830] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Jul 7 15:41:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:56.497373 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee763953-8b6f-4b0a-9961-a6dc60a7fd37 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:56.497667 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee763953-8b6f-4b0a-9961-a6dc60a7fd37 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:56.497953 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee763953-8b6f-4b0a-9961-a6dc60a7fd37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:56.524377 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:56.524377 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-ee763953-8b6f-4b0a-9961-a6dc60a7fd37 None None] MFA Rules not processed for user `03f8ed4d3d5a4447ba2c61b3dd4c314a`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:41:56.524377 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-ee763953-8b6f-4b0a-9961-a6dc60a7fd37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:56.535228 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1403/2831] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:41:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:41:56.700941 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9becc3-72ec-4e85-9d6f-2124a05e1157 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:56.702575 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-7c9becc3-72ec-4e85-9d6f-2124a05e1157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:56.776628 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=100 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:41:56.831617 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9becc3-72ec-4e85-9d6f-2124a05e1157 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:56.839986 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd40a8-487d-4176-a6c9-c351c455067b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b18033f849b240999d12c1885d2f7b91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:56.840942 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd40a8-487d-4176-a6c9-c351c455067b service swift] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:56.841231 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd40a8-487d-4176-a6c9-c351c455067b service swift] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:56.841836 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd40a8-487d-4176-a6c9-c351c455067b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:56.853511 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fd40a8-487d-4176-a6c9-c351c455067b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:56.855182 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fd40a8-487d-4176-a6c9-c351c455067b service swift] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:41:56.867458 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1429/2832] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Jul 7 15:41:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 07 15:41:59.778534 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee763953-8b6f-4b0a-9961-a6dc60a7fd37 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:41:59.785177 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-ee763953-8b6f-4b0a-9961-a6dc60a7fd37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:41:59.905964 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee763953-8b6f-4b0a-9961-a6dc60a7fd37 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:41:59.915831 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77434bee-bcc7-492c-8c00-e2ea7b00daab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:41:59.916797 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77434bee-bcc7-492c-8c00-e2ea7b00daab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:41:59.917154 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77434bee-bcc7-492c-8c00-e2ea7b00daab admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:41:59.917608 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77434bee-bcc7-492c-8c00-e2ea7b00daab admin admin] PATH_INFO: `/v3/users/4a7231c86e1a4c4bb851f317a935079d` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:41:59.919378 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77434bee-bcc7-492c-8c00-e2ea7b00daab admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a7231c86e1a4c4bb851f317a935079d)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:41:59.921298 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77434bee-bcc7-492c-8c00-e2ea7b00daab admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:42:00.136979 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-77434bee-bcc7-492c-8c00-e2ea7b00daab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a7231c86e1a4c4bb851f317a935079d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:42:00.152237 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-77434bee-bcc7-492c-8c00-e2ea7b00daab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a7231c86e1a4c4bb851f317a935079d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:42:00.167847 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-77434bee-bcc7-492c-8c00-e2ea7b00daab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a7231c86e1a4c4bb851f317a935079d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:42:00.168674 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-77434bee-bcc7-492c-8c00-e2ea7b00daab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a7231c86e1a4c4bb851f317a935079d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:42:00.288896 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-77434bee-bcc7-492c-8c00-e2ea7b00daab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a7231c86e1a4c4bb851f317a935079d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:42:00.298980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1404/2833] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:41:59 2022] DELETE /identity/v3/users/4a7231c86e1a4c4bb851f317a935079d => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:42:00.666502 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd40a8-487d-4176-a6c9-c351c455067b service swift] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:42:00.711747 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd40a8-487d-4176-a6c9-c351c455067b service swift] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:42:00.833518 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a53c4-c978-43db-8cc6-76a2c4bdee14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:42:00.834894 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a53c4-c978-43db-8cc6-76a2c4bdee14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:42:00.835498 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a53c4-c978-43db-8cc6-76a2c4bdee14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:42:00.836082 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a53c4-c978-43db-8cc6-76a2c4bdee14 admin admin] PATH_INFO: `/v3/projects/41be827d61314cdbb5387c24e6353bb5` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:42:00.837648 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0a53c4-c978-43db-8cc6-76a2c4bdee14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41be827d61314cdbb5387c24e6353bb5)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:42:00.839862 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0a53c4-c978-43db-8cc6-76a2c4bdee14 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:42:01.281897 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-7e0a53c4-c978-43db-8cc6-76a2c4bdee14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41be827d61314cdbb5387c24e6353bb5', 'request_id': 'req-7e0a53c4-c978-43db-8cc6-76a2c4bdee14'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:42:01.299480 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-7e0a53c4-c978-43db-8cc6-76a2c4bdee14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41be827d61314cdbb5387c24e6353bb5', 'request_id': 'req-7e0a53c4-c978-43db-8cc6-76a2c4bdee14'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:42:01.316196 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-7e0a53c4-c978-43db-8cc6-76a2c4bdee14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41be827d61314cdbb5387c24e6353bb5', 'request_id': 'req-7e0a53c4-c978-43db-8cc6-76a2c4bdee14'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:42:01.406243 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-7e0a53c4-c978-43db-8cc6-76a2c4bdee14 admin admin] The token cache is being invalidate because project 41be827d61314cdbb5387c24e6353bb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:42:01.406534 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-7e0a53c4-c978-43db-8cc6-76a2c4bdee14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:42:01.408041 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-7e0a53c4-c978-43db-8cc6-76a2c4bdee14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:42:01.410186 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1430/2834] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:42:00 2022] DELETE /identity/v3/projects/41be827d61314cdbb5387c24e6353bb5 => generated 0 bytes in 748 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:42:01.414555 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77434bee-bcc7-492c-8c00-e2ea7b00daab admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:42:01.416216 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-77434bee-bcc7-492c-8c00-e2ea7b00daab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:42:01.439178 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77434bee-bcc7-492c-8c00-e2ea7b00daab admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:42:01.579600 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0a25ab-4ff2-4606-a059-e8caaaa285cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:42:01.580676 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0a25ab-4ff2-4606-a059-e8caaaa285cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:42:01.581110 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0a25ab-4ff2-4606-a059-e8caaaa285cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:42:01.581492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0a25ab-4ff2-4606-a059-e8caaaa285cd admin admin] PATH_INFO: `/v3/users/17bd76c88a434eb2b79cac3e491430b4` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:42:01.583129 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0a25ab-4ff2-4606-a059-e8caaaa285cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=17bd76c88a434eb2b79cac3e491430b4)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:42:01.588703 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0a25ab-4ff2-4606-a059-e8caaaa285cd admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:42:01.620131 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:42:01.845498 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-6c0a25ab-4ff2-4606-a059-e8caaaa285cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17bd76c88a434eb2b79cac3e491430b4'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:42:01.857633 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-6c0a25ab-4ff2-4606-a059-e8caaaa285cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17bd76c88a434eb2b79cac3e491430b4'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:42:01.883661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-6c0a25ab-4ff2-4606-a059-e8caaaa285cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17bd76c88a434eb2b79cac3e491430b4'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:42:01.884410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-6c0a25ab-4ff2-4606-a059-e8caaaa285cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17bd76c88a434eb2b79cac3e491430b4'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:42:01.963379 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-6c0a25ab-4ff2-4606-a059-e8caaaa285cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17bd76c88a434eb2b79cac3e491430b4'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:42:01.972348 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1405/2835] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:42:01 2022] DELETE /identity/v3/users/17bd76c88a434eb2b79cac3e491430b4 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:42:02.382458 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a53c4-c978-43db-8cc6-76a2c4bdee14 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:42:02.384698 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-7e0a53c4-c978-43db-8cc6-76a2c4bdee14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:42:02.411679 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a53c4-c978-43db-8cc6-76a2c4bdee14 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:42:02.508959 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:42:02.524602 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054eb164-fd44-40d8-a9ee-df80388a2fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:42:02.528401 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054eb164-fd44-40d8-a9ee-df80388a2fbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:42:02.529259 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054eb164-fd44-40d8-a9ee-df80388a2fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:42:02.529615 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054eb164-fd44-40d8-a9ee-df80388a2fbf admin admin] PATH_INFO: `/v3/projects/7714a76c80c94191a77576e343e64e16` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:42:02.533170 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054eb164-fd44-40d8-a9ee-df80388a2fbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=7714a76c80c94191a77576e343e64e16)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:42:02.537389 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054eb164-fd44-40d8-a9ee-df80388a2fbf admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:42:02.710060 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-054eb164-fd44-40d8-a9ee-df80388a2fbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7714a76c80c94191a77576e343e64e16', 'request_id': 'req-054eb164-fd44-40d8-a9ee-df80388a2fbf'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:42:02.719337 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-054eb164-fd44-40d8-a9ee-df80388a2fbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7714a76c80c94191a77576e343e64e16', 'request_id': 'req-054eb164-fd44-40d8-a9ee-df80388a2fbf'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:42:02.726879 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-054eb164-fd44-40d8-a9ee-df80388a2fbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7714a76c80c94191a77576e343e64e16', 'request_id': 'req-054eb164-fd44-40d8-a9ee-df80388a2fbf'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:42:02.859622 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-054eb164-fd44-40d8-a9ee-df80388a2fbf admin admin] The token cache is being invalidate because project 7714a76c80c94191a77576e343e64e16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:42:02.859622 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-054eb164-fd44-40d8-a9ee-df80388a2fbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:42:02.860399 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-054eb164-fd44-40d8-a9ee-df80388a2fbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:42:02.866936 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1431/2836] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:42:02 2022] DELETE /identity/v3/projects/7714a76c80c94191a77576e343e64e16 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:42:08.605210 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df7450-98c9-4bc2-9270-36765cb8b88d None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:42:08.605671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df7450-98c9-4bc2-9270-36765cb8b88d None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:42:08.605771 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df7450-98c9-4bc2-9270-36765cb8b88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:42:08.630205 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-51df7450-98c9-4bc2-9270-36765cb8b88d None None] MFA Rules not processed for user `975faecf991a41f4b116537b39e04d3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:42:08.632634 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-51df7450-98c9-4bc2-9270-36765cb8b88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:42:08.688028 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1406/2837] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:42:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:42:12.855930 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=27,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:42:12.880653 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce23d5b-e0a4-4441-a753-266f89a72083 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:42:12.880943 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce23d5b-e0a4-4441-a753-266f89a72083 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:42:12.881277 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce23d5b-e0a4-4441-a753-266f89a72083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:42:12.900511 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-dce23d5b-e0a4-4441-a753-266f89a72083 None None] MFA Rules not processed for user `cd3c04f39d774998b66e6ce920925691`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:42:12.901724 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-dce23d5b-e0a4-4441-a753-266f89a72083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:42:12.965713 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1432/2838] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Jul 7 15:42:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:42:12.977527 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df7450-98c9-4bc2-9270-36765cb8b88d None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:42:12.979011 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-51df7450-98c9-4bc2-9270-36765cb8b88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:42:13.062005 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df7450-98c9-4bc2-9270-36765cb8b88d None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:42:13.067642 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b85453f-dd27-46e2-9418-04a18525d4a4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b18033f849b240999d12c1885d2f7b91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:42:13.068228 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85453f-dd27-46e2-9418-04a18525d4a4 service swift] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:42:13.068516 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85453f-dd27-46e2-9418-04a18525d4a4 service swift] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:42:13.068798 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85453f-dd27-46e2-9418-04a18525d4a4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:42:13.078600 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b85453f-dd27-46e2-9418-04a18525d4a4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:42:13.080271 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b85453f-dd27-46e2-9418-04a18525d4a4 service swift] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:42:13.089908 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1407/2839] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Jul 7 15:42:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 07 15:42:18.825229 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce23d5b-e0a4-4441-a753-266f89a72083 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:42:18.826845 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-dce23d5b-e0a4-4441-a753-266f89a72083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:42:18.932009 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce23d5b-e0a4-4441-a753-266f89a72083 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:42:18.936249 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1828c27-1262-4566-b28e-43c506b62a7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:42:18.936977 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1828c27-1262-4566-b28e-43c506b62a7d service cinder] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:42:18.937323 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1828c27-1262-4566-b28e-43c506b62a7d service cinder] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:42:18.937661 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1828c27-1262-4566-b28e-43c506b62a7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:42:18.939558 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-c1828c27-1262-4566-b28e-43c506b62a7d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:42:19.048079 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1828c27-1262-4566-b28e-43c506b62a7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:42:19.049218 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1828c27-1262-4566-b28e-43c506b62a7d service cinder] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:42:19.063387 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1433/2840] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:42:18 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 15:42:23.085710 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:DELETE=7,keystone:SELECT=77,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:42:24.815111 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1624c01-0d4c-4990-bad3-ebe9b9e993c8 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:42:24.822405 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1624c01-0d4c-4990-bad3-ebe9b9e993c8 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:42:24.822405 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1624c01-0d4c-4990-bad3-ebe9b9e993c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:42:24.835391 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-e1624c01-0d4c-4990-bad3-ebe9b9e993c8 None None] MFA Rules not processed for user `03f8ed4d3d5a4447ba2c61b3dd4c314a`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:42:24.836665 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-e1624c01-0d4c-4990-bad3-ebe9b9e993c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:42:24.894367 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1408/2841] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:42:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:42:27.591716 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e58fd0-fe4c-473d-af86-838add2eab1c None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:42:27.592365 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e58fd0-fe4c-473d-af86-838add2eab1c None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:42:27.592365 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e58fd0-fe4c-473d-af86-838add2eab1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:42:27.609612 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-e3e58fd0-fe4c-473d-af86-838add2eab1c None None] MFA Rules not processed for user `05e0f1ddeb224677b18cfc769dc45d47`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:42:27.610801 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-e3e58fd0-fe4c-473d-af86-838add2eab1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:42:27.617921 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1434/2842] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:42:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:42:34.887621 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=25 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:42:37.604833 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=76 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:42:44.125673 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1624c01-0d4c-4990-bad3-ebe9b9e993c8 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:42:44.127141 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-e1624c01-0d4c-4990-bad3-ebe9b9e993c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:42:44.184308 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1624c01-0d4c-4990-bad3-ebe9b9e993c8 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:42:44.188161 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705318fc-c689-4661-a0f5-f893d8ef1eb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:42:44.188910 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705318fc-c689-4661-a0f5-f893d8ef1eb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:42:44.189292 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705318fc-c689-4661-a0f5-f893d8ef1eb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:42:44.189664 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705318fc-c689-4661-a0f5-f893d8ef1eb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:42:44.191550 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-705318fc-c689-4661-a0f5-f893d8ef1eb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:42:44.261946 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705318fc-c689-4661-a0f5-f893d8ef1eb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:42:44.263363 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705318fc-c689-4661-a0f5-f893d8ef1eb2 service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:42:44.273318 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1409/2843] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:42:44 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:42:49.615233 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5495455-615c-45c9-9e61-40ef8c5f9f4d None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:42:49.615957 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5495455-615c-45c9-9e61-40ef8c5f9f4d None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:42:49.615957 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5495455-615c-45c9-9e61-40ef8c5f9f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:42:49.635060 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-a5495455-615c-45c9-9e61-40ef8c5f9f4d None None] MFA Rules not processed for user `cd3c04f39d774998b66e6ce920925691`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:42:49.636342 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-a5495455-615c-45c9-9e61-40ef8c5f9f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:42:49.643907 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1435/2844] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Jul 7 15:42:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:42:49.656636 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705318fc-c689-4661-a0f5-f893d8ef1eb2 service neutron] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:42:49.669228 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705318fc-c689-4661-a0f5-f893d8ef1eb2 service neutron] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:42:49.674486 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6662f1-23fe-43d3-bc9c-c241fc68133a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b18033f849b240999d12c1885d2f7b91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:42:49.675122 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6662f1-23fe-43d3-bc9c-c241fc68133a service swift] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:42:49.675410 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6662f1-23fe-43d3-bc9c-c241fc68133a service swift] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:42:49.675739 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6662f1-23fe-43d3-bc9c-c241fc68133a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:42:49.684088 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6662f1-23fe-43d3-bc9c-c241fc68133a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:42:49.685667 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6662f1-23fe-43d3-bc9c-c241fc68133a service swift] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:42:49.693681 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1410/2845] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Jul 7 15:42:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 07 15:42:50.707612 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5495455-615c-45c9-9e61-40ef8c5f9f4d None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:42:50.710518 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-a5495455-615c-45c9-9e61-40ef8c5f9f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:42:50.733336 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5495455-615c-45c9-9e61-40ef8c5f9f4d None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:42:50.742247 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd65994-54f0-44d5-ac8a-f3addd7113a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:42:50.743270 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd65994-54f0-44d5-ac8a-f3addd7113a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:42:50.743658 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd65994-54f0-44d5-ac8a-f3addd7113a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:42:50.743961 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd65994-54f0-44d5-ac8a-f3addd7113a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:42:50.745988 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-3cd65994-54f0-44d5-ac8a-f3addd7113a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:42:50.848329 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd65994-54f0-44d5-ac8a-f3addd7113a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:42:50.849789 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd65994-54f0-44d5-ac8a-f3addd7113a2 service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:42:50.866391 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1436/2846] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:42:50 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:42:59.690497 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=56 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:43:00.859155 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=33 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:43:06.073627 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355bcaeb-0727-4197-b1a6-12b17ebb2eaf None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:43:06.074134 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355bcaeb-0727-4197-b1a6-12b17ebb2eaf None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:43:06.074244 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355bcaeb-0727-4197-b1a6-12b17ebb2eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:43:06.093165 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-355bcaeb-0727-4197-b1a6-12b17ebb2eaf None None] MFA Rules not processed for user `cd3c04f39d774998b66e6ce920925691`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:43:06.094341 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-355bcaeb-0727-4197-b1a6-12b17ebb2eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:43:06.101359 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1411/2847] 10.222.0.25 () {58 vars in 967 bytes} [Thu Jul 7 15:43:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:43:06.113504 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd65994-54f0-44d5-ac8a-f3addd7113a2 service neutron] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:43:06.124620 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd65994-54f0-44d5-ac8a-f3addd7113a2 service neutron] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:43:06.129519 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106fa1b8-6c5b-4012-9369-8783d0dbb60c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b18033f849b240999d12c1885d2f7b91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:43:06.130150 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fa1b8-6c5b-4012-9369-8783d0dbb60c service swift] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:43:06.130434 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fa1b8-6c5b-4012-9369-8783d0dbb60c service swift] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:43:06.130712 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fa1b8-6c5b-4012-9369-8783d0dbb60c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:43:06.139142 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106fa1b8-6c5b-4012-9369-8783d0dbb60c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:43:06.140727 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106fa1b8-6c5b-4012-9369-8783d0dbb60c service swift] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:43:06.149240 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1437/2848] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Jul 7 15:43:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 07 15:43:08.894000 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355bcaeb-0727-4197-b1a6-12b17ebb2eaf None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:43:08.895594 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-355bcaeb-0727-4197-b1a6-12b17ebb2eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:43:08.962113 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355bcaeb-0727-4197-b1a6-12b17ebb2eaf None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:43:08.966676 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32b2b2-1bcc-48b0-b415-030d9caf92d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:43:08.967678 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32b2b2-1bcc-48b0-b415-030d9caf92d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:43:08.967844 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32b2b2-1bcc-48b0-b415-030d9caf92d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:43:08.968119 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32b2b2-1bcc-48b0-b415-030d9caf92d5 admin admin] PATH_INFO: `/v3/users/03f8ed4d3d5a4447ba2c61b3dd4c314a` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:43:08.969323 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc32b2b2-1bcc-48b0-b415-030d9caf92d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03f8ed4d3d5a4447ba2c61b3dd4c314a)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:43:08.971236 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc32b2b2-1bcc-48b0-b415-030d9caf92d5 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:43:09.169072 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-bc32b2b2-1bcc-48b0-b415-030d9caf92d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03f8ed4d3d5a4447ba2c61b3dd4c314a'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:43:09.176200 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-bc32b2b2-1bcc-48b0-b415-030d9caf92d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03f8ed4d3d5a4447ba2c61b3dd4c314a'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:43:09.187970 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-bc32b2b2-1bcc-48b0-b415-030d9caf92d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03f8ed4d3d5a4447ba2c61b3dd4c314a'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:43:09.188643 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-bc32b2b2-1bcc-48b0-b415-030d9caf92d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03f8ed4d3d5a4447ba2c61b3dd4c314a'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:43:09.258996 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-bc32b2b2-1bcc-48b0-b415-030d9caf92d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03f8ed4d3d5a4447ba2c61b3dd4c314a'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:43:09.265146 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1412/2849] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:43:08 2022] DELETE /identity/v3/users/03f8ed4d3d5a4447ba2c61b3dd4c314a => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:43:09.541076 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106fa1b8-6c5b-4012-9369-8783d0dbb60c service swift] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:43:09.553356 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106fa1b8-6c5b-4012-9369-8783d0dbb60c service swift] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:43:09.610074 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee9c4aa-3b8b-4969-a3c7-bf806cf485c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:43:09.610923 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9c4aa-3b8b-4969-a3c7-bf806cf485c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:43:09.611203 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9c4aa-3b8b-4969-a3c7-bf806cf485c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:43:09.611475 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9c4aa-3b8b-4969-a3c7-bf806cf485c2 admin admin] PATH_INFO: `/v3/projects/f90b44820e424210b4dea0abc23ea469` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:43:09.612686 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee9c4aa-3b8b-4969-a3c7-bf806cf485c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f90b44820e424210b4dea0abc23ea469)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:43:09.614848 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee9c4aa-3b8b-4969-a3c7-bf806cf485c2 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:43:09.704421 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-8ee9c4aa-3b8b-4969-a3c7-bf806cf485c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f90b44820e424210b4dea0abc23ea469', 'request_id': 'req-8ee9c4aa-3b8b-4969-a3c7-bf806cf485c2'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:43:09.709967 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-8ee9c4aa-3b8b-4969-a3c7-bf806cf485c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f90b44820e424210b4dea0abc23ea469', 'request_id': 'req-8ee9c4aa-3b8b-4969-a3c7-bf806cf485c2'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:43:09.716321 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-8ee9c4aa-3b8b-4969-a3c7-bf806cf485c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f90b44820e424210b4dea0abc23ea469', 'request_id': 'req-8ee9c4aa-3b8b-4969-a3c7-bf806cf485c2'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:43:09.805250 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-8ee9c4aa-3b8b-4969-a3c7-bf806cf485c2 admin admin] The token cache is being invalidate because project f90b44820e424210b4dea0abc23ea469 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:43:09.805459 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-8ee9c4aa-3b8b-4969-a3c7-bf806cf485c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:43:09.806051 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-8ee9c4aa-3b8b-4969-a3c7-bf806cf485c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:43:09.807657 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1438/2850] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:43:09 2022] DELETE /identity/v3/projects/f90b44820e424210b4dea0abc23ea469 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:43:09.812676 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde8294-6611-4d61-b83e-76d72ad87209 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:43:09.812968 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde8294-6611-4d61-b83e-76d72ad87209 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:43:09.813244 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde8294-6611-4d61-b83e-76d72ad87209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:43:09.831894 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-4fde8294-6611-4d61-b83e-76d72ad87209 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:43:09.834632 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-4fde8294-6611-4d61-b83e-76d72ad87209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:43:09.890880 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1413/2851] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:43:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:43:09.898488 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5cfc76-ce7d-46f4-ae5b-fe63b1b021ab None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:43:09.898782 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5cfc76-ce7d-46f4-ae5b-fe63b1b021ab None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:43:09.899061 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5cfc76-ce7d-46f4-ae5b-fe63b1b021ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:43:09.918775 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-2c5cfc76-ce7d-46f4-ae5b-fe63b1b021ab None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:43:09.920044 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-2c5cfc76-ce7d-46f4-ae5b-fe63b1b021ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:43:09.927933 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1439/2852] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:43:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:43:09.932625 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fde8294-6611-4d61-b83e-76d72ad87209 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:43:09.942376 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fde8294-6611-4d61-b83e-76d72ad87209 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:43:09.946985 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5a815-587d-450c-a6b6-11cf65f27657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:43:09.947674 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5a815-587d-450c-a6b6-11cf65f27657 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:43:09.947955 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5a815-587d-450c-a6b6-11cf65f27657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:43:09.948226 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5a815-587d-450c-a6b6-11cf65f27657 admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:43:09.948947 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df5a815-587d-450c-a6b6-11cf65f27657 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:43:09.949248 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df5a815-587d-450c-a6b6-11cf65f27657 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:43:09.950851 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df5a815-587d-450c-a6b6-11cf65f27657 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:43:09.959550 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:43:09.962946 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1414/2853] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:43:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:43:09.969402 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5cfc76-ce7d-46f4-ae5b-fe63b1b021ab None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:43:09.979358 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5cfc76-ce7d-46f4-ae5b-fe63b1b021ab None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:43:09.984109 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a3b48-5abc-48f1-954a-585a92bef033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:43:09.984751 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a3b48-5abc-48f1-954a-585a92bef033 admin admin] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:43:09.985033 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a3b48-5abc-48f1-954a-585a92bef033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:43:09.985309 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a3b48-5abc-48f1-954a-585a92bef033 admin admin] PATH_INFO: `/v3/projects` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:43:09.986002 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a3b48-5abc-48f1-954a-585a92bef033 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:43:09.987919 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a3b48-5abc-48f1-954a-585a92bef033 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:43:10.082603 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1440/2854] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:43:09 2022] POST /identity/v3/projects => generated 371 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:43:10.086944 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5a815-587d-450c-a6b6-11cf65f27657 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:43:10.099006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5a815-587d-450c-a6b6-11cf65f27657 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:43:10.154885 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff68132-d3eb-4925-8731-46d2ed3188be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:43:10.155535 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff68132-d3eb-4925-8731-46d2ed3188be admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:43:10.155849 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff68132-d3eb-4925-8731-46d2ed3188be admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:43:10.156124 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff68132-d3eb-4925-8731-46d2ed3188be admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:43:10.156799 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff68132-d3eb-4925-8731-46d2ed3188be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:43:10.158693 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff68132-d3eb-4925-8731-46d2ed3188be admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:43:10.247079 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1415/2855] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:43:10 2022] POST /identity/v3/users => generated 286 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 15:43:10.251268 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a3b48-5abc-48f1-954a-585a92bef033 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:43:10.261813 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a3b48-5abc-48f1-954a-585a92bef033 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:43:10.266581 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563116d6-3cf7-45ef-b2cf-95a94344e2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:43:10.268791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563116d6-3cf7-45ef-b2cf-95a94344e2f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:43:10.268791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563116d6-3cf7-45ef-b2cf-95a94344e2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:43:10.268791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563116d6-3cf7-45ef-b2cf-95a94344e2f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:43:10.268791 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563116d6-3cf7-45ef-b2cf-95a94344e2f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:43:10.270182 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563116d6-3cf7-45ef-b2cf-95a94344e2f6 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:43:10.281082 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1441/2856] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:43:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:43:10.284983 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff68132-d3eb-4925-8731-46d2ed3188be admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:43:10.295629 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff68132-d3eb-4925-8731-46d2ed3188be admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:43:10.300116 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d4d33c-4106-42d4-af50-76033f5b80a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:43:10.300759 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4d33c-4106-42d4-af50-76033f5b80a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:43:10.301037 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4d33c-4106-42d4-af50-76033f5b80a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:43:10.301314 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4d33c-4106-42d4-af50-76033f5b80a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:43:10.302035 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d4d33c-4106-42d4-af50-76033f5b80a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:43:10.303661 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d4d33c-4106-42d4-af50-76033f5b80a7 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:43:10.313406 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1416/2857] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:43:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:43:10.317543 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563116d6-3cf7-45ef-b2cf-95a94344e2f6 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:43:10.327817 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563116d6-3cf7-45ef-b2cf-95a94344e2f6 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:43:10.332547 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de775e-d1b5-4e39-b6a2-87f727427637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:43:10.333465 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de775e-d1b5-4e39-b6a2-87f727427637 admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:43:10.333699 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de775e-d1b5-4e39-b6a2-87f727427637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:43:10.333973 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de775e-d1b5-4e39-b6a2-87f727427637 admin admin] PATH_INFO: `/v3/projects/48854562fb004693a0c5d50039ef4eee/users/e3dbd06335ee49348ffd1c215a02cc2d/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:43:10.357525 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67de775e-d1b5-4e39-b6a2-87f727427637 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48854562fb004693a0c5d50039ef4eee, user_id=e3dbd06335ee49348ffd1c215a02cc2d, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:43:10.360222 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67de775e-d1b5-4e39-b6a2-87f727427637 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:43:10.412461 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1442/2858] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:43:10 2022] PUT /identity/v3/projects/48854562fb004693a0c5d50039ef4eee/users/e3dbd06335ee49348ffd1c215a02cc2d/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:43:10.426215 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0dd78-5464-411a-98b8-fab51199e7fb None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:43:10.426507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0dd78-5464-411a-98b8-fab51199e7fb None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:43:10.426809 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0dd78-5464-411a-98b8-fab51199e7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:43:10.453543 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-75c0dd78-5464-411a-98b8-fab51199e7fb None None] MFA Rules not processed for user `e3dbd06335ee49348ffd1c215a02cc2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:43:10.454752 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-75c0dd78-5464-411a-98b8-fab51199e7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:43:10.517197 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1417/2859] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:43:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:43:10.528759 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de775e-d1b5-4e39-b6a2-87f727427637 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:43:10.534141 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-67de775e-d1b5-4e39-b6a2-87f727427637 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:43:10.545415 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:43:10.589452 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de775e-d1b5-4e39-b6a2-87f727427637 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:43:10.593112 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d3dd9-8c3b-4a62-9a89-f7a101746a53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:43:10.593755 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d3dd9-8c3b-4a62-9a89-f7a101746a53 service cinder] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:43:10.594033 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d3dd9-8c3b-4a62-9a89-f7a101746a53 service cinder] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:43:10.594332 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d3dd9-8c3b-4a62-9a89-f7a101746a53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:43:10.603018 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d3dd9-8c3b-4a62-9a89-f7a101746a53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:43:10.604472 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d3dd9-8c3b-4a62-9a89-f7a101746a53 service cinder] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:43:10.613067 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1443/2860] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:43:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:43:10.636814 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c0dd78-5464-411a-98b8-fab51199e7fb None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:43:10.699089 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c0dd78-5464-411a-98b8-fab51199e7fb None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:43:10.703581 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04487e70-f9c9-4b1c-8500-919660d1da40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:43:10.704427 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04487e70-f9c9-4b1c-8500-919660d1da40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:43:10.704705 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04487e70-f9c9-4b1c-8500-919660d1da40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:43:10.704975 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04487e70-f9c9-4b1c-8500-919660d1da40 admin admin] PATH_INFO: `/v3/users/e3dbd06335ee49348ffd1c215a02cc2d` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:43:10.706153 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04487e70-f9c9-4b1c-8500-919660d1da40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3dbd06335ee49348ffd1c215a02cc2d)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:43:10.708093 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04487e70-f9c9-4b1c-8500-919660d1da40 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:43:10.718962 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:43:10.872546 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-04487e70-f9c9-4b1c-8500-919660d1da40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3dbd06335ee49348ffd1c215a02cc2d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:43:10.879237 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-04487e70-f9c9-4b1c-8500-919660d1da40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3dbd06335ee49348ffd1c215a02cc2d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:43:10.891048 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-04487e70-f9c9-4b1c-8500-919660d1da40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3dbd06335ee49348ffd1c215a02cc2d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:43:10.891758 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-04487e70-f9c9-4b1c-8500-919660d1da40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3dbd06335ee49348ffd1c215a02cc2d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:43:10.973633 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-04487e70-f9c9-4b1c-8500-919660d1da40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3dbd06335ee49348ffd1c215a02cc2d'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:43:10.980930 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1418/2861] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:43:10 2022] DELETE /identity/v3/users/e3dbd06335ee49348ffd1c215a02cc2d => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:43:10.985490 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c865d39-9608-4aee-9538-e12eea952cb6 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:43:10.985774 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c865d39-9608-4aee-9538-e12eea952cb6 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:43:10.986047 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c865d39-9608-4aee-9538-e12eea952cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:43:11.003158 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-2c865d39-9608-4aee-9538-e12eea952cb6 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:43:11.005885 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-2c865d39-9608-4aee-9538-e12eea952cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:43:11.062288 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1444/2862] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:43:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:43:11.071835 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04487e70-f9c9-4b1c-8500-919660d1da40 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:43:11.073325 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-04487e70-f9c9-4b1c-8500-919660d1da40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:43:11.130671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04487e70-f9c9-4b1c-8500-919660d1da40 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:43:11.134171 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47032168-f40f-44c3-880d-b2ddefff6074 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:43:11.134803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47032168-f40f-44c3-880d-b2ddefff6074 service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:43:11.135085 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47032168-f40f-44c3-880d-b2ddefff6074 service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:43:11.135359 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47032168-f40f-44c3-880d-b2ddefff6074 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:43:11.144202 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47032168-f40f-44c3-880d-b2ddefff6074 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:43:11.145606 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47032168-f40f-44c3-880d-b2ddefff6074 service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:43:11.155490 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1419/2863] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:43:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:43:11.563677 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c865d39-9608-4aee-9538-e12eea952cb6 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:43:11.565080 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-2c865d39-9608-4aee-9538-e12eea952cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:43:11.579016 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c865d39-9608-4aee-9538-e12eea952cb6 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:43:11.584717 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104f4846-f7e9-45a8-9d4e-eb10cd9a4604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:43:11.585585 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f4846-f7e9-45a8-9d4e-eb10cd9a4604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:43:11.586516 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f4846-f7e9-45a8-9d4e-eb10cd9a4604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:43:11.586809 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f4846-f7e9-45a8-9d4e-eb10cd9a4604 admin admin] PATH_INFO: `/v3/projects/48854562fb004693a0c5d50039ef4eee` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:43:11.588090 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104f4846-f7e9-45a8-9d4e-eb10cd9a4604 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48854562fb004693a0c5d50039ef4eee)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:43:11.590038 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104f4846-f7e9-45a8-9d4e-eb10cd9a4604 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:43:11.692878 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-104f4846-f7e9-45a8-9d4e-eb10cd9a4604 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48854562fb004693a0c5d50039ef4eee', 'request_id': 'req-104f4846-f7e9-45a8-9d4e-eb10cd9a4604'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:43:11.698975 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-104f4846-f7e9-45a8-9d4e-eb10cd9a4604 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48854562fb004693a0c5d50039ef4eee', 'request_id': 'req-104f4846-f7e9-45a8-9d4e-eb10cd9a4604'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:43:11.705211 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-104f4846-f7e9-45a8-9d4e-eb10cd9a4604 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48854562fb004693a0c5d50039ef4eee', 'request_id': 'req-104f4846-f7e9-45a8-9d4e-eb10cd9a4604'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:43:11.789403 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-104f4846-f7e9-45a8-9d4e-eb10cd9a4604 admin admin] The token cache is being invalidate because project 48854562fb004693a0c5d50039ef4eee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:43:11.789801 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-104f4846-f7e9-45a8-9d4e-eb10cd9a4604 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:43:11.790482 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-104f4846-f7e9-45a8-9d4e-eb10cd9a4604 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:43:11.792144 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1445/2864] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:43:11 2022] DELETE /identity/v3/projects/48854562fb004693a0c5d50039ef4eee => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:43:11.797158 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e9f10-3357-4e53-809d-c0891c665fb9 None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:43:11.797524 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e9f10-3357-4e53-809d-c0891c665fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:43:11.797803 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e9f10-3357-4e53-809d-c0891c665fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:43:11.815915 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-4e1e9f10-3357-4e53-809d-c0891c665fb9 None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:43:11.817116 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-4e1e9f10-3357-4e53-809d-c0891c665fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:43:11.878827 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1420/2865] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:43:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:43:11.951427 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187d209-1793-4f1f-a295-d3b7ff06314d None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:43:11.951656 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187d209-1793-4f1f-a295-d3b7ff06314d None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:43:11.951770 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187d209-1793-4f1f-a295-d3b7ff06314d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:43:11.973362 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-0187d209-1793-4f1f-a295-d3b7ff06314d None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:43:11.974594 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-0187d209-1793-4f1f-a295-d3b7ff06314d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:43:11.984886 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1446/2866] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:43:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:43:11.988011 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1e9f10-3357-4e53-809d-c0891c665fb9 None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:43:11.992455 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0187d209-1793-4f1f-a295-d3b7ff06314d None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:43:11.996666 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-0187d209-1793-4f1f-a295-d3b7ff06314d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:43:11.998199 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1e9f10-3357-4e53-809d-c0891c665fb9 None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:43:12.002918 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345845e0-6bcd-48db-a814-ec70df5602bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:43:12.003617 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345845e0-6bcd-48db-a814-ec70df5602bf admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:43:12.003909 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345845e0-6bcd-48db-a814-ec70df5602bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:43:12.004277 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345845e0-6bcd-48db-a814-ec70df5602bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:43:12.005006 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345845e0-6bcd-48db-a814-ec70df5602bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 15:43:12.005313 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345845e0-6bcd-48db-a814-ec70df5602bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:43:12.006954 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345845e0-6bcd-48db-a814-ec70df5602bf admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:43:12.021261 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1421/2867] 10.222.0.25 () {58 vars in 1152 bytes} [Thu Jul 7 15:43:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 15:43:12.025670 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345845e0-6bcd-48db-a814-ec70df5602bf admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:43:12.036995 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345845e0-6bcd-48db-a814-ec70df5602bf admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:43:12.038496 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:43:12.043539 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06304f-f6d3-4584-a5e7-884fc8ec33f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:43:12.043539 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06304f-f6d3-4584-a5e7-884fc8ec33f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:43:12.043835 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06304f-f6d3-4584-a5e7-884fc8ec33f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:43:12.043932 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06304f-f6d3-4584-a5e7-884fc8ec33f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:43:12.044564 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d06304f-f6d3-4584-a5e7-884fc8ec33f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:43:12.046537 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d06304f-f6d3-4584-a5e7-884fc8ec33f4 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:43:12.062815 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:43:12.063057 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0187d209-1793-4f1f-a295-d3b7ff06314d None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:43:12.066835 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c49f72-c9a0-4d6f-bb51-287a920a4ce2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:43:12.067483 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c49f72-c9a0-4d6f-bb51-287a920a4ce2 service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:43:12.067795 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c49f72-c9a0-4d6f-bb51-287a920a4ce2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:43:12.068070 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c49f72-c9a0-4d6f-bb51-287a920a4ce2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:43:12.071758 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-70c49f72-c9a0-4d6f-bb51-287a920a4ce2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:43:12.110750 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1422/2868] 10.222.0.25 () {60 vars in 1152 bytes} [Thu Jul 7 15:43:12 2022] POST /identity/v3/projects => generated 383 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:43:12.111767 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06304f-f6d3-4584-a5e7-884fc8ec33f4 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:43:12.153020 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c49f72-c9a0-4d6f-bb51-287a920a4ce2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:43:12.154473 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c49f72-c9a0-4d6f-bb51-287a920a4ce2 service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:43:12.168806 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1447/2869] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:43:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:43:12.181822 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06304f-f6d3-4584-a5e7-884fc8ec33f4 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:43:12.192501 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc6e0b-20b1-42b6-b490-d8532637aa6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:43:12.193192 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc6e0b-20b1-42b6-b490-d8532637aa6c admin admin] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:43:12.193514 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc6e0b-20b1-42b6-b490-d8532637aa6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:43:12.193757 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc6e0b-20b1-42b6-b490-d8532637aa6c admin admin] PATH_INFO: `/v3/users` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:43:12.194454 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cc6e0b-20b1-42b6-b490-d8532637aa6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:43:12.197085 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cc6e0b-20b1-42b6-b490-d8532637aa6c admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:43:12.294276 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1423/2870] 10.222.0.25 () {60 vars in 1143 bytes} [Thu Jul 7 15:43:12 2022] POST /identity/v3/users => generated 292 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 15:43:12.298589 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c49f72-c9a0-4d6f-bb51-287a920a4ce2 service neutron] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:43:12.311414 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c49f72-c9a0-4d6f-bb51-287a920a4ce2 service neutron] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:43:12.316137 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06022e60-8b16-4620-a90f-7b69103d216f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:43:12.316809 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06022e60-8b16-4620-a90f-7b69103d216f admin admin] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:43:12.317097 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06022e60-8b16-4620-a90f-7b69103d216f admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:43:12.317380 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06022e60-8b16-4620-a90f-7b69103d216f admin admin] PATH_INFO: `/v3/roles` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:43:12.318111 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06022e60-8b16-4620-a90f-7b69103d216f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:43:12.319736 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06022e60-8b16-4620-a90f-7b69103d216f admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:43:12.330894 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1448/2871] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:43:12.334948 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc6e0b-20b1-42b6-b490-d8532637aa6c admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:43:12.345901 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc6e0b-20b1-42b6-b490-d8532637aa6c admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:43:12.350903 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891c018-c02c-427b-9ada-99d8d9abc219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:43:12.351724 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891c018-c02c-427b-9ada-99d8d9abc219 admin admin] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:43:12.352071 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891c018-c02c-427b-9ada-99d8d9abc219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:43:12.352439 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891c018-c02c-427b-9ada-99d8d9abc219 admin admin] PATH_INFO: `/v3/roles` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:43:12.353263 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6891c018-c02c-427b-9ada-99d8d9abc219 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:43:12.354877 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6891c018-c02c-427b-9ada-99d8d9abc219 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:43:12.365713 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1424/2872] 10.222.0.25 () {58 vars in 1121 bytes} [Thu Jul 7 15:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 15:43:12.369972 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06022e60-8b16-4620-a90f-7b69103d216f admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:43:12.379862 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06022e60-8b16-4620-a90f-7b69103d216f admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:43:12.384956 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b485e-c742-4dfd-b20e-306c80b9c94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:43:12.385833 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b485e-c742-4dfd-b20e-306c80b9c94e admin admin] REQUEST_METHOD: `PUT` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:43:12.386141 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b485e-c742-4dfd-b20e-306c80b9c94e admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:43:12.386418 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b485e-c742-4dfd-b20e-306c80b9c94e admin admin] PATH_INFO: `/v3/projects/d087fee0c67c47f98f424fbdb919b547/users/fedbf09a8fe146709e18f019a18854c6/roles/d43d0d189e254500875f5f86e6abd178` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:43:12.402531 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404b485e-c742-4dfd-b20e-306c80b9c94e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d087fee0c67c47f98f424fbdb919b547, user_id=fedbf09a8fe146709e18f019a18854c6, role_id=d43d0d189e254500875f5f86e6abd178)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:43:12.405257 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404b485e-c742-4dfd-b20e-306c80b9c94e admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:43:12.505567 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1449/2873] 10.222.0.25 () {60 vars in 1482 bytes} [Thu Jul 7 15:43:12 2022] PUT /identity/v3/projects/d087fee0c67c47f98f424fbdb919b547/users/fedbf09a8fe146709e18f019a18854c6/roles/d43d0d189e254500875f5f86e6abd178 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:43:12.511244 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8e298-bfc9-47b7-ad71-2c26ca38ab0c None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:43:12.511540 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8e298-bfc9-47b7-ad71-2c26ca38ab0c None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:43:12.511989 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8e298-bfc9-47b7-ad71-2c26ca38ab0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:43:12.529610 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-54c8e298-bfc9-47b7-ad71-2c26ca38ab0c None None] MFA Rules not processed for user `30014035e7d641d5aecffdbe884d3d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:43:12.530853 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-54c8e298-bfc9-47b7-ad71-2c26ca38ab0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:43:12.593825 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1425/2874] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:43:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:43:12.603938 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b485e-c742-4dfd-b20e-306c80b9c94e admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:43:12.661185 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b485e-c742-4dfd-b20e-306c80b9c94e admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:43:12.665068 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25114bf-88a0-48fe-8ae5-0db414e4e785 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:43:12.665751 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25114bf-88a0-48fe-8ae5-0db414e4e785 service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:43:12.666030 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25114bf-88a0-48fe-8ae5-0db414e4e785 service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:43:12.666303 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25114bf-88a0-48fe-8ae5-0db414e4e785 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:43:12.675678 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25114bf-88a0-48fe-8ae5-0db414e4e785 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:43:12.679497 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25114bf-88a0-48fe-8ae5-0db414e4e785 service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:43:12.687508 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1450/2875] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:43:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:43:13.400424 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8e298-bfc9-47b7-ad71-2c26ca38ab0c None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:43:13.410492 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8e298-bfc9-47b7-ad71-2c26ca38ab0c None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:43:13.414747 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee2f6c9-ab0e-4311-a11e-54ef2b5e8281 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:43:13.415475 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee2f6c9-ab0e-4311-a11e-54ef2b5e8281 service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:43:13.416184 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee2f6c9-ab0e-4311-a11e-54ef2b5e8281 service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:43:13.416184 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee2f6c9-ab0e-4311-a11e-54ef2b5e8281 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:43:13.418070 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-fee2f6c9-ab0e-4311-a11e-54ef2b5e8281 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:43:13.509996 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee2f6c9-ab0e-4311-a11e-54ef2b5e8281 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:43:13.511427 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee2f6c9-ab0e-4311-a11e-54ef2b5e8281 service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:43:13.517975 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:43:13.524166 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1426/2876] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:43:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:43:14.316326 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e066d-688b-4bf2-a6a4-ab49c2ba8486 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:43:14.316792 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e066d-688b-4bf2-a6a4-ab49c2ba8486 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:43:14.317039 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e066d-688b-4bf2-a6a4-ab49c2ba8486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:43:14.345153 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-445e066d-688b-4bf2-a6a4-ab49c2ba8486 None None] MFA Rules not processed for user `fedbf09a8fe146709e18f019a18854c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:43:14.351934 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-445e066d-688b-4bf2-a6a4-ab49c2ba8486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:43:14.376538 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:43:14.412063 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1451/2877] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:43:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:43:14.430373 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee2f6c9-ab0e-4311-a11e-54ef2b5e8281 service neutron] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:43:14.431878 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-fee2f6c9-ab0e-4311-a11e-54ef2b5e8281 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:43:14.494288 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee2f6c9-ab0e-4311-a11e-54ef2b5e8281 service neutron] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:43:14.497940 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64248424-8813-49a5-8d12-e9d11b805314 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0fbebb3dd2458ba4fe5d7f0ce211b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:43:14.498605 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64248424-8813-49a5-8d12-e9d11b805314 service cinder] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:43:14.498892 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64248424-8813-49a5-8d12-e9d11b805314 service cinder] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:43:14.499168 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64248424-8813-49a5-8d12-e9d11b805314 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:43:14.509181 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64248424-8813-49a5-8d12-e9d11b805314 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:43:14.510777 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64248424-8813-49a5-8d12-e9d11b805314 service cinder] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:43:14.520732 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1427/2878] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:43:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:43:16.149215 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a15bc-a220-4009-993a-83c0243c5b4d None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:43:16.149706 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a15bc-a220-4009-993a-83c0243c5b4d None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:43:16.149805 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a15bc-a220-4009-993a-83c0243c5b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:43:16.167268 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-7f5a15bc-a220-4009-993a-83c0243c5b4d None None] MFA Rules not processed for user `fedbf09a8fe146709e18f019a18854c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:43:16.168489 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-7f5a15bc-a220-4009-993a-83c0243c5b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:43:16.175221 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1452/2879] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:43:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:43:24.516443 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=26 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:43:26.161555 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=16 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:44:05.365235 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64248424-8813-49a5-8d12-e9d11b805314 service cinder] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:44:05.366886 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-64248424-8813-49a5-8d12-e9d11b805314 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:44:05.380795 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64248424-8813-49a5-8d12-e9d11b805314 service cinder] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:44:05.385319 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e772a0e-5222-439b-b729-40224d5e8ced service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:44:05.386051 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e772a0e-5222-439b-b729-40224d5e8ced service neutron] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:44:05.387431 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e772a0e-5222-439b-b729-40224d5e8ced service neutron] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:44:05.387431 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e772a0e-5222-439b-b729-40224d5e8ced service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:44:05.388862 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-5e772a0e-5222-439b-b729-40224d5e8ced service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:44:05.399113 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e772a0e-5222-439b-b729-40224d5e8ced service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:44:05.400633 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e772a0e-5222-439b-b729-40224d5e8ced service neutron] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:44:05.410630 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1428/2880] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:44:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 15:44:12.699858 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a15bc-a220-4009-993a-83c0243c5b4d None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:44:12.702666 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-7f5a15bc-a220-4009-993a-83c0243c5b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:44:12.719478 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a15bc-a220-4009-993a-83c0243c5b4d None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:44:12.725211 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7b1a9-e538-49d7-844e-fadada92bafd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:44:12.730810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7b1a9-e538-49d7-844e-fadada92bafd service nova] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:44:12.730810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7b1a9-e538-49d7-844e-fadada92bafd service nova] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:44:12.730810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7b1a9-e538-49d7-844e-fadada92bafd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:44:12.730810 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-8af7b1a9-e538-49d7-844e-fadada92bafd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:44:12.750266 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af7b1a9-e538-49d7-844e-fadada92bafd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:44:12.752086 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af7b1a9-e538-49d7-844e-fadada92bafd service nova] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:44:12.763267 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1453/2881] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:44:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:44:15.406711 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=6 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:44:22.758509 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=6 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:44:25.164281 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d5833-8f89-4229-9934-d9029752768f None None] REQUEST_METHOD: `POST` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:44:25.164281 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d5833-8f89-4229-9934-d9029752768f None None] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:44:25.164281 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d5833-8f89-4229-9934-d9029752768f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:44:25.182073 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.auth.core [None req-163d5833-8f89-4229-9934-d9029752768f None None] MFA Rules not processed for user `fedbf09a8fe146709e18f019a18854c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:44:25.183296 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-163d5833-8f89-4229-9934-d9029752768f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:44:25.190780 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1429/2882] 10.222.0.25 () {58 vars in 957 bytes} [Thu Jul 7 15:44:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:44:25.433234 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7b1a9-e538-49d7-844e-fadada92bafd service nova] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:44:25.443464 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7b1a9-e538-49d7-844e-fadada92bafd service nova] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:44:25.447127 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cc6425-33bd-4878-9adc-7509b610ee46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:44:25.447804 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc6425-33bd-4878-9adc-7509b610ee46 service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:44:25.448065 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc6425-33bd-4878-9adc-7509b610ee46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:44:25.448335 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc6425-33bd-4878-9adc-7509b610ee46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:44:25.450080 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-b7cc6425-33bd-4878-9adc-7509b610ee46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:44:25.459527 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cc6425-33bd-4878-9adc-7509b610ee46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:44:25.460957 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cc6425-33bd-4878-9adc-7509b610ee46 service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:44:25.468965 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1454/2883] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:44:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:44:25.865008 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d5833-8f89-4229-9934-d9029752768f None None] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:44:25.866588 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-163d5833-8f89-4229-9934-d9029752768f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:44:25.941447 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d5833-8f89-4229-9934-d9029752768f None None] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:44:25.946312 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115a42c-3201-4ebe-a4a2-3456ffaf6953 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '755c2782cfb042f89950b8fc862e4fa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:44:25.946955 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115a42c-3201-4ebe-a4a2-3456ffaf6953 service placement] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:44:25.947237 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115a42c-3201-4ebe-a4a2-3456ffaf6953 service placement] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:44:25.947527 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115a42c-3201-4ebe-a4a2-3456ffaf6953 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:44:25.949348 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-6115a42c-3201-4ebe-a4a2-3456ffaf6953 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:44:25.970408 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6115a42c-3201-4ebe-a4a2-3456ffaf6953 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:44:25.972039 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6115a42c-3201-4ebe-a4a2-3456ffaf6953 service placement] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:44:25.982397 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1430/2884] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:44:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:44:28.419307 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cc6425-33bd-4878-9adc-7509b610ee46 service neutron] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:44:28.422115 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-b7cc6425-33bd-4878-9adc-7509b610ee46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:44:28.453460 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cc6425-33bd-4878-9adc-7509b610ee46 service neutron] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:44:28.473286 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24d5059-5e7f-4695-94a5-0398e725a023 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:44:28.474257 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d5059-5e7f-4695-94a5-0398e725a023 service nova] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:44:28.474614 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d5059-5e7f-4695-94a5-0398e725a023 service nova] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:44:28.474895 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d5059-5e7f-4695-94a5-0398e725a023 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:44:28.479709 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-f24d5059-5e7f-4695-94a5-0398e725a023 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:44:28.499757 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24d5059-5e7f-4695-94a5-0398e725a023 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:44:28.502307 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24d5059-5e7f-4695-94a5-0398e725a023 service nova] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:44:28.517859 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1455/2885] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:44:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:44:35.978047 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=35 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:44:37.185740 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115a42c-3201-4ebe-a4a2-3456ffaf6953 service placement] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:44:37.198949 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115a42c-3201-4ebe-a4a2-3456ffaf6953 service placement] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:44:37.204669 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee0f954-1800-4e74-b074-b92ab0a2699c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de1632555cb345f2b9213da1925c02f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:44:37.205374 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0f954-1800-4e74-b074-b92ab0a2699c service nova] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:44:37.205653 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0f954-1800-4e74-b074-b92ab0a2699c service nova] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:44:37.205930 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0f954-1800-4e74-b074-b92ab0a2699c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:44:37.207764 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-2ee0f954-1800-4e74-b074-b92ab0a2699c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:44:37.225540 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee0f954-1800-4e74-b074-b92ab0a2699c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:44:37.227158 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee0f954-1800-4e74-b074-b92ab0a2699c service nova] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:44:37.240703 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1431/2886] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:44:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:44:38.511324 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=12 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:44:40.543780 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a2ff4-8c93-4a05-b9ea-2af8b8443c27 None None] REQUEST_METHOD: `POST` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:44:40.544406 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a2ff4-8c93-4a05-b9ea-2af8b8443c27 None None] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:44:40.544406 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a2ff4-8c93-4a05-b9ea-2af8b8443c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:44:40.563769 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.auth.core [None req-9a8a2ff4-8c93-4a05-b9ea-2af8b8443c27 None None] MFA Rules not processed for user `cd3c04f39d774998b66e6ce920925691`. Rule list: `[]` (Enabled: `True`). {{(pid=80301) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 15:44:40.565051 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-9a8a2ff4-8c93-4a05-b9ea-2af8b8443c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:44:40.620637 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1456/2887] 10.222.0.25 () {58 vars in 967 bytes} [Thu Jul 7 15:44:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 15:44:40.759342 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee0f954-1800-4e74-b074-b92ab0a2699c service nova] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:44:40.760356 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-2ee0f954-1800-4e74-b074-b92ab0a2699c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:44:40.828694 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee0f954-1800-4e74-b074-b92ab0a2699c service nova] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:44:40.834063 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93980754-a28e-49df-8598-b66644ebab63 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b18033f849b240999d12c1885d2f7b91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:44:40.834731 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93980754-a28e-49df-8598-b66644ebab63 service swift] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:44:40.835012 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93980754-a28e-49df-8598-b66644ebab63 service swift] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:44:40.835285 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93980754-a28e-49df-8598-b66644ebab63 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:44:40.843320 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93980754-a28e-49df-8598-b66644ebab63 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:44:40.845599 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93980754-a28e-49df-8598-b66644ebab63 service swift] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:44:40.853611 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1432/2888] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Jul 7 15:44:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 07 15:44:41.567743 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a2ff4-8c93-4a05-b9ea-2af8b8443c27 None None] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:44:41.577136 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a2ff4-8c93-4a05-b9ea-2af8b8443c27 None None] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:44:41.581467 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414bb1c-a1e0-449c-ab0b-a1b4e9969e54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e79efaea155c46ecb9831956e68aec8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:44:41.582165 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414bb1c-a1e0-449c-ab0b-a1b4e9969e54 service neutron] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:44:41.582449 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414bb1c-a1e0-449c-ab0b-a1b4e9969e54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:44:41.582725 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414bb1c-a1e0-449c-ab0b-a1b4e9969e54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:44:41.584465 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-5414bb1c-a1e0-449c-ab0b-a1b4e9969e54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:44:41.596407 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5414bb1c-a1e0-449c-ab0b-a1b4e9969e54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:44:41.597893 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5414bb1c-a1e0-449c-ab0b-a1b4e9969e54 service neutron] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:44:41.608532 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1457/2889] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:44:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:44:45.906893 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93980754-a28e-49df-8598-b66644ebab63 service swift] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:44:45.911807 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-93980754-a28e-49df-8598-b66644ebab63 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:44:45.923278 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93980754-a28e-49df-8598-b66644ebab63 service swift] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:44:45.928416 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e99f3-25bb-49bb-8ee5-a4dc0bfa5b34 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '755c2782cfb042f89950b8fc862e4fa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:44:45.929049 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e99f3-25bb-49bb-8ee5-a4dc0bfa5b34 service placement] REQUEST_METHOD: `GET` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:44:45.929330 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e99f3-25bb-49bb-8ee5-a4dc0bfa5b34 service placement] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:44:45.930142 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e99f3-25bb-49bb-8ee5-a4dc0bfa5b34 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:44:45.931921 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-749e99f3-25bb-49bb-8ee5-a4dc0bfa5b34 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:44:45.944104 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e99f3-25bb-49bb-8ee5-a4dc0bfa5b34 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:44:45.945707 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e99f3-25bb-49bb-8ee5-a4dc0bfa5b34 service placement] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:44:45.960490 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1433/2890] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:44:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:44:49.897876 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414bb1c-a1e0-449c-ab0b-a1b4e9969e54 service neutron] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:44:49.899666 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-5414bb1c-a1e0-449c-ab0b-a1b4e9969e54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:44:49.917327 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414bb1c-a1e0-449c-ab0b-a1b4e9969e54 service neutron] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:44:49.922160 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81823950-111c-4145-8e70-2de6014ffd07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:44:49.923029 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81823950-111c-4145-8e70-2de6014ffd07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:44:49.923495 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81823950-111c-4145-8e70-2de6014ffd07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:44:49.923817 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81823950-111c-4145-8e70-2de6014ffd07 admin admin] PATH_INFO: `/v3/users/05e0f1ddeb224677b18cfc769dc45d47` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:44:49.925003 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81823950-111c-4145-8e70-2de6014ffd07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05e0f1ddeb224677b18cfc769dc45d47)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:44:49.926954 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81823950-111c-4145-8e70-2de6014ffd07 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:44:50.099698 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-81823950-111c-4145-8e70-2de6014ffd07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05e0f1ddeb224677b18cfc769dc45d47'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:50.100290 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-81823950-111c-4145-8e70-2de6014ffd07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05e0f1ddeb224677b18cfc769dc45d47'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:50.107234 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-81823950-111c-4145-8e70-2de6014ffd07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05e0f1ddeb224677b18cfc769dc45d47'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:50.107928 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-81823950-111c-4145-8e70-2de6014ffd07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05e0f1ddeb224677b18cfc769dc45d47'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:50.120460 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-81823950-111c-4145-8e70-2de6014ffd07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05e0f1ddeb224677b18cfc769dc45d47'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:50.153337 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e99f3-25bb-49bb-8ee5-a4dc0bfa5b34 service placement] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:44:50.154759 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-749e99f3-25bb-49bb-8ee5-a4dc0bfa5b34 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:44:50.172246 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e99f3-25bb-49bb-8ee5-a4dc0bfa5b34 service placement] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:44:50.175907 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefee366-d330-4062-90ea-3c35ea978f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:44:50.176783 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefee366-d330-4062-90ea-3c35ea978f55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:44:50.177079 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefee366-d330-4062-90ea-3c35ea978f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:44:50.177356 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefee366-d330-4062-90ea-3c35ea978f55 admin admin] PATH_INFO: `/v3/users/975faecf991a41f4b116537b39e04d3c` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:44:50.182941 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefee366-d330-4062-90ea-3c35ea978f55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=975faecf991a41f4b116537b39e04d3c)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:44:50.185504 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefee366-d330-4062-90ea-3c35ea978f55 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:44:50.189923 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1458/2891] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:44:49 2022] DELETE /identity/v3/users/05e0f1ddeb224677b18cfc769dc45d47 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:44:50.317552 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-aefee366-d330-4062-90ea-3c35ea978f55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '975faecf991a41f4b116537b39e04d3c'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:50.324990 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-aefee366-d330-4062-90ea-3c35ea978f55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '975faecf991a41f4b116537b39e04d3c'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:50.337535 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-aefee366-d330-4062-90ea-3c35ea978f55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '975faecf991a41f4b116537b39e04d3c'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:50.338360 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-aefee366-d330-4062-90ea-3c35ea978f55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '975faecf991a41f4b116537b39e04d3c'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:50.388919 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-aefee366-d330-4062-90ea-3c35ea978f55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '975faecf991a41f4b116537b39e04d3c'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:50.395252 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1434/2892] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:44:50 2022] DELETE /identity/v3/users/975faecf991a41f4b116537b39e04d3c => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:44:50.422306 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81823950-111c-4145-8e70-2de6014ffd07 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:44:50.423903 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-81823950-111c-4145-8e70-2de6014ffd07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:44:50.503170 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81823950-111c-4145-8e70-2de6014ffd07 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:44:50.511217 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:44:50.561591 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d44f14-e117-43c6-b5dc-bfeab077fdb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:44:50.562509 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d44f14-e117-43c6-b5dc-bfeab077fdb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:44:50.562793 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d44f14-e117-43c6-b5dc-bfeab077fdb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:44:50.563066 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d44f14-e117-43c6-b5dc-bfeab077fdb9 admin admin] PATH_INFO: `/v3/projects/b72dae7a2e644460b4c63da884023c82` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:44:50.564428 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d44f14-e117-43c6-b5dc-bfeab077fdb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b72dae7a2e644460b4c63da884023c82)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:44:50.566327 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d44f14-e117-43c6-b5dc-bfeab077fdb9 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:44:50.665861 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefee366-d330-4062-90ea-3c35ea978f55 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:44:50.667685 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-aefee366-d330-4062-90ea-3c35ea978f55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:44:50.680074 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-f7d44f14-e117-43c6-b5dc-bfeab077fdb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b72dae7a2e644460b4c63da884023c82', 'request_id': 'req-f7d44f14-e117-43c6-b5dc-bfeab077fdb9'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:50.685770 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-f7d44f14-e117-43c6-b5dc-bfeab077fdb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b72dae7a2e644460b4c63da884023c82', 'request_id': 'req-f7d44f14-e117-43c6-b5dc-bfeab077fdb9'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:50.687743 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefee366-d330-4062-90ea-3c35ea978f55 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:44:50.693682 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-f7d44f14-e117-43c6-b5dc-bfeab077fdb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b72dae7a2e644460b4c63da884023c82', 'request_id': 'req-f7d44f14-e117-43c6-b5dc-bfeab077fdb9'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:50.722135 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:44:50.746132 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed56519e-e9f5-4c6d-924c-f38630164173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:44:50.746970 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed56519e-e9f5-4c6d-924c-f38630164173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:44:50.747296 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed56519e-e9f5-4c6d-924c-f38630164173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:44:50.747584 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed56519e-e9f5-4c6d-924c-f38630164173 admin admin] PATH_INFO: `/v3/projects/0b2770b1b56d4f0ea8c3c1759a8e504f` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:44:50.748760 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed56519e-e9f5-4c6d-924c-f38630164173 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b2770b1b56d4f0ea8c3c1759a8e504f)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:44:50.750688 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed56519e-e9f5-4c6d-924c-f38630164173 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:44:50.754478 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-f7d44f14-e117-43c6-b5dc-bfeab077fdb9 admin admin] The token cache is being invalidate because project b72dae7a2e644460b4c63da884023c82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:44:50.754783 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-f7d44f14-e117-43c6-b5dc-bfeab077fdb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:50.760873 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-f7d44f14-e117-43c6-b5dc-bfeab077fdb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:50.760873 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1459/2893] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:44:50 2022] DELETE /identity/v3/projects/b72dae7a2e644460b4c63da884023c82 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:44:50.761110 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d44f14-e117-43c6-b5dc-bfeab077fdb9 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:44:50.762266 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-f7d44f14-e117-43c6-b5dc-bfeab077fdb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:44:50.775272 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d44f14-e117-43c6-b5dc-bfeab077fdb9 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:44:50.779921 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4e20b5-5bde-445d-9898-cafad8aa4c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:44:50.784631 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e20b5-5bde-445d-9898-cafad8aa4c48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:44:50.784631 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e20b5-5bde-445d-9898-cafad8aa4c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:44:50.784631 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e20b5-5bde-445d-9898-cafad8aa4c48 admin admin] PATH_INFO: `/v3/users/840981beb61a41e79b70ee6ffff3e511` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:44:50.784631 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4e20b5-5bde-445d-9898-cafad8aa4c48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=840981beb61a41e79b70ee6ffff3e511)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:44:50.785103 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4e20b5-5bde-445d-9898-cafad8aa4c48 admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:44:50.840668 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-ed56519e-e9f5-4c6d-924c-f38630164173 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b2770b1b56d4f0ea8c3c1759a8e504f', 'request_id': 'req-ed56519e-e9f5-4c6d-924c-f38630164173'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:50.846861 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-ed56519e-e9f5-4c6d-924c-f38630164173 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b2770b1b56d4f0ea8c3c1759a8e504f', 'request_id': 'req-ed56519e-e9f5-4c6d-924c-f38630164173'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:50.853160 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-ed56519e-e9f5-4c6d-924c-f38630164173 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b2770b1b56d4f0ea8c3c1759a8e504f', 'request_id': 'req-ed56519e-e9f5-4c6d-924c-f38630164173'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:50.913811 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-ed56519e-e9f5-4c6d-924c-f38630164173 admin admin] The token cache is being invalidate because project 0b2770b1b56d4f0ea8c3c1759a8e504f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:44:50.914015 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-ed56519e-e9f5-4c6d-924c-f38630164173 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:50.914610 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-ed56519e-e9f5-4c6d-924c-f38630164173 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:50.916179 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1435/2894] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:44:50 2022] DELETE /identity/v3/projects/0b2770b1b56d4f0ea8c3c1759a8e504f => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:44:50.920190 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed56519e-e9f5-4c6d-924c-f38630164173 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:44:50.921454 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-ed56519e-e9f5-4c6d-924c-f38630164173 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:44:50.980114 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-1a4e20b5-5bde-445d-9898-cafad8aa4c48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '840981beb61a41e79b70ee6ffff3e511'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:50.980788 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-1a4e20b5-5bde-445d-9898-cafad8aa4c48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '840981beb61a41e79b70ee6ffff3e511'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:50.985845 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed56519e-e9f5-4c6d-924c-f38630164173 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:44:50.991322 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267449b1-4c65-4df5-a7ac-8a1a410a8c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:44:50.992220 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267449b1-4c65-4df5-a7ac-8a1a410a8c90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:44:50.992507 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267449b1-4c65-4df5-a7ac-8a1a410a8c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:44:50.992785 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267449b1-4c65-4df5-a7ac-8a1a410a8c90 admin admin] PATH_INFO: `/v3/users/3c2aa80580734f5ebeaa96e7a2e39a0b` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:44:50.994032 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267449b1-4c65-4df5-a7ac-8a1a410a8c90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c2aa80580734f5ebeaa96e7a2e39a0b)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:44:50.995980 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267449b1-4c65-4df5-a7ac-8a1a410a8c90 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:44:51.001826 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-1a4e20b5-5bde-445d-9898-cafad8aa4c48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '840981beb61a41e79b70ee6ffff3e511'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:51.011880 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-1a4e20b5-5bde-445d-9898-cafad8aa4c48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '840981beb61a41e79b70ee6ffff3e511'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:51.022769 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-1a4e20b5-5bde-445d-9898-cafad8aa4c48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '840981beb61a41e79b70ee6ffff3e511'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:51.097744 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1460/2895] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:44:50 2022] DELETE /identity/v3/users/840981beb61a41e79b70ee6ffff3e511 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:44:51.170621 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-267449b1-4c65-4df5-a7ac-8a1a410a8c90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c2aa80580734f5ebeaa96e7a2e39a0b'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:51.176528 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-267449b1-4c65-4df5-a7ac-8a1a410a8c90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c2aa80580734f5ebeaa96e7a2e39a0b'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:51.184671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:44:51.190451 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-267449b1-4c65-4df5-a7ac-8a1a410a8c90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c2aa80580734f5ebeaa96e7a2e39a0b'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:51.191372 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-267449b1-4c65-4df5-a7ac-8a1a410a8c90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c2aa80580734f5ebeaa96e7a2e39a0b'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:51.297115 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-267449b1-4c65-4df5-a7ac-8a1a410a8c90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c2aa80580734f5ebeaa96e7a2e39a0b'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:51.303744 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1436/2896] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:44:50 2022] DELETE /identity/v3/users/3c2aa80580734f5ebeaa96e7a2e39a0b => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 15:44:51.340345 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4e20b5-5bde-445d-9898-cafad8aa4c48 admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:44:51.341956 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-1a4e20b5-5bde-445d-9898-cafad8aa4c48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:44:51.357534 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:44:51.414485 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4e20b5-5bde-445d-9898-cafad8aa4c48 admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:44:51.472054 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b4e0e7-a889-4da9-9abb-bbadeae25fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:44:51.473271 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b4e0e7-a889-4da9-9abb-bbadeae25fcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:44:51.473555 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b4e0e7-a889-4da9-9abb-bbadeae25fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:44:51.473826 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b4e0e7-a889-4da9-9abb-bbadeae25fcb admin admin] PATH_INFO: `/v3/projects/98953ecda1544abdb7652065dafa75b3` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:44:51.475441 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b4e0e7-a889-4da9-9abb-bbadeae25fcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=98953ecda1544abdb7652065dafa75b3)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:44:51.477401 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b4e0e7-a889-4da9-9abb-bbadeae25fcb admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:44:51.567686 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267449b1-4c65-4df5-a7ac-8a1a410a8c90 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:44:51.569304 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-267449b1-4c65-4df5-a7ac-8a1a410a8c90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:44:51.581524 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-e4b4e0e7-a889-4da9-9abb-bbadeae25fcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98953ecda1544abdb7652065dafa75b3', 'request_id': 'req-e4b4e0e7-a889-4da9-9abb-bbadeae25fcb'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:51.587133 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-e4b4e0e7-a889-4da9-9abb-bbadeae25fcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98953ecda1544abdb7652065dafa75b3', 'request_id': 'req-e4b4e0e7-a889-4da9-9abb-bbadeae25fcb'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:51.592812 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267449b1-4c65-4df5-a7ac-8a1a410a8c90 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:44:51.594839 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-e4b4e0e7-a889-4da9-9abb-bbadeae25fcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98953ecda1544abdb7652065dafa75b3', 'request_id': 'req-e4b4e0e7-a889-4da9-9abb-bbadeae25fcb'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:51.654138 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdaa6a3-3045-4a09-b60f-7bdd8e6f9ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:44:51.655045 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdaa6a3-3045-4a09-b60f-7bdd8e6f9ad9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:44:51.655345 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdaa6a3-3045-4a09-b60f-7bdd8e6f9ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:44:51.655938 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdaa6a3-3045-4a09-b60f-7bdd8e6f9ad9 admin admin] PATH_INFO: `/v3/projects/0d68f8aa14cf4e96b936c100efc198f2` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:44:51.660470 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdaa6a3-3045-4a09-b60f-7bdd8e6f9ad9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d68f8aa14cf4e96b936c100efc198f2)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:44:51.660470 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdaa6a3-3045-4a09-b60f-7bdd8e6f9ad9 admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:44:51.689098 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-e4b4e0e7-a889-4da9-9abb-bbadeae25fcb admin admin] The token cache is being invalidate because project 98953ecda1544abdb7652065dafa75b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80301) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:44:51.689098 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-e4b4e0e7-a889-4da9-9abb-bbadeae25fcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:51.689098 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-e4b4e0e7-a889-4da9-9abb-bbadeae25fcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:51.689098 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1461/2897] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:44:51 2022] DELETE /identity/v3/projects/98953ecda1544abdb7652065dafa75b3 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:44:51.785394 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-ebdaa6a3-3045-4a09-b60f-7bdd8e6f9ad9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d68f8aa14cf4e96b936c100efc198f2', 'request_id': 'req-ebdaa6a3-3045-4a09-b60f-7bdd8e6f9ad9'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:51.791894 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-ebdaa6a3-3045-4a09-b60f-7bdd8e6f9ad9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d68f8aa14cf4e96b936c100efc198f2', 'request_id': 'req-ebdaa6a3-3045-4a09-b60f-7bdd8e6f9ad9'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:51.799508 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-ebdaa6a3-3045-4a09-b60f-7bdd8e6f9ad9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d68f8aa14cf4e96b936c100efc198f2', 'request_id': 'req-ebdaa6a3-3045-4a09-b60f-7bdd8e6f9ad9'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:51.869275 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-ebdaa6a3-3045-4a09-b60f-7bdd8e6f9ad9 admin admin] The token cache is being invalidate because project 0d68f8aa14cf4e96b936c100efc198f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:44:51.869671 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-ebdaa6a3-3045-4a09-b60f-7bdd8e6f9ad9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:51.870299 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-ebdaa6a3-3045-4a09-b60f-7bdd8e6f9ad9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:44:51.871950 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1437/2898] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:44:51 2022] DELETE /identity/v3/projects/0d68f8aa14cf4e96b936c100efc198f2 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:45:01.681977 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=69,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:45:01.865580 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=55,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:45:32.399733 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b4e0e7-a889-4da9-9abb-bbadeae25fcb admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:45:32.401361 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-e4b4e0e7-a889-4da9-9abb-bbadeae25fcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:45:32.414411 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b4e0e7-a889-4da9-9abb-bbadeae25fcb admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:45:32.470619 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa96b8-43ed-4309-8300-2cf7617bf01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:45:32.471487 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa96b8-43ed-4309-8300-2cf7617bf01a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:45:32.471799 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa96b8-43ed-4309-8300-2cf7617bf01a admin admin] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:45:32.472095 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa96b8-43ed-4309-8300-2cf7617bf01a admin admin] PATH_INFO: `/v3/users/fedbf09a8fe146709e18f019a18854c6` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:45:32.473279 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daa96b8-43ed-4309-8300-2cf7617bf01a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fedbf09a8fe146709e18f019a18854c6)` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:45:32.475187 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daa96b8-43ed-4309-8300-2cf7617bf01a admin admin] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:45:32.623456 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-5daa96b8-43ed-4309-8300-2cf7617bf01a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fedbf09a8fe146709e18f019a18854c6'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:45:32.624129 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-5daa96b8-43ed-4309-8300-2cf7617bf01a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fedbf09a8fe146709e18f019a18854c6'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:45:32.631042 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-5daa96b8-43ed-4309-8300-2cf7617bf01a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fedbf09a8fe146709e18f019a18854c6'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:45:32.631712 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-5daa96b8-43ed-4309-8300-2cf7617bf01a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fedbf09a8fe146709e18f019a18854c6'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:45:32.643822 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.notifications [None req-5daa96b8-43ed-4309-8300-2cf7617bf01a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fedbf09a8fe146709e18f019a18854c6'} {{(pid=80301) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:45:32.759932 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1462/2899] 10.222.0.25 () {58 vars in 1223 bytes} [Thu Jul 7 15:45:32 2022] DELETE /identity/v3/users/fedbf09a8fe146709e18f019a18854c6 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:45:33.001409 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdaa6a3-3045-4a09-b60f-7bdd8e6f9ad9 admin admin] Authenticating user token {{(pid=80300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:45:33.002912 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.fernet_utils [None req-ebdaa6a3-3045-4a09-b60f-7bdd8e6f9ad9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:45:33.016218 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdaa6a3-3045-4a09-b60f-7bdd8e6f9ad9 admin admin] Validating token access rules against request {{(pid=80300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:45:33.075475 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b08a0b3-4c72-4586-bf27-16bb5ed058dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30014035e7d641d5aecffdbe884d3d52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28133f9a98be4af88ed357f76f5a4a2e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:45:33.076360 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08a0b3-4c72-4586-bf27-16bb5ed058dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:45:33.076639 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08a0b3-4c72-4586-bf27-16bb5ed058dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:45:33.076913 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08a0b3-4c72-4586-bf27-16bb5ed058dc admin admin] PATH_INFO: `/v3/projects/d087fee0c67c47f98f424fbdb919b547` {{(pid=80300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:45:33.078070 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b08a0b3-4c72-4586-bf27-16bb5ed058dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=d087fee0c67c47f98f424fbdb919b547)` {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:45:33.080069 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b08a0b3-4c72-4586-bf27-16bb5ed058dc admin admin] RBAC: Authorization granted {{(pid=80300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:45:33.176124 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-8b08a0b3-4c72-4586-bf27-16bb5ed058dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd087fee0c67c47f98f424fbdb919b547', 'request_id': 'req-8b08a0b3-4c72-4586-bf27-16bb5ed058dc'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:45:33.181775 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-8b08a0b3-4c72-4586-bf27-16bb5ed058dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd087fee0c67c47f98f424fbdb919b547', 'request_id': 'req-8b08a0b3-4c72-4586-bf27-16bb5ed058dc'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:45:33.188016 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-8b08a0b3-4c72-4586-bf27-16bb5ed058dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd087fee0c67c47f98f424fbdb919b547', 'request_id': 'req-8b08a0b3-4c72-4586-bf27-16bb5ed058dc'} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:45:33.244236 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-8b08a0b3-4c72-4586-bf27-16bb5ed058dc admin admin] The token cache is being invalidate because project d087fee0c67c47f98f424fbdb919b547 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 15:45:33.244525 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-8b08a0b3-4c72-4586-bf27-16bb5ed058dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:45:33.245138 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG keystone.notifications [None req-8b08a0b3-4c72-4586-bf27-16bb5ed058dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 15:45:33.246701 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: [pid: 80300|app: 0|req: 1438/2900] 10.222.0.25 () {58 vars in 1232 bytes} [Thu Jul 7 15:45:33 2022] DELETE /identity/v3/projects/d087fee0c67c47f98f424fbdb919b547 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 15:45:42.755950 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:45:43.241111 dv-cinder-841302-5-iscsi devstack@keystone.service[80300]: DEBUG dbcounter [-] [80300] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 15:46:13.559994 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa96b8-43ed-4309-8300-2cf7617bf01a admin admin] Authenticating user token {{(pid=80301) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 15:46:13.561426 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-5daa96b8-43ed-4309-8300-2cf7617bf01a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:46:13.710386 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa96b8-43ed-4309-8300-2cf7617bf01a admin admin] Validating token access rules against request {{(pid=80301) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 15:46:13.768529 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85f56bb-9a7d-4225-860d-436b23adf0b2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '755c2782cfb042f89950b8fc862e4fa1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5260c7fd3614e9e93b167c86a94260e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80301) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 15:46:13.769166 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f56bb-9a7d-4225-860d-436b23adf0b2 service placement] REQUEST_METHOD: `GET` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 15:46:13.769445 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f56bb-9a7d-4225-860d-436b23adf0b2 service placement] SCRIPT_NAME: `/identity` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 15:46:13.769717 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f56bb-9a7d-4225-860d-436b23adf0b2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80301) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 15:46:13.771482 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.fernet_utils [None req-e85f56bb-9a7d-4225-860d-436b23adf0b2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80301) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 15:46:13.783023 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85f56bb-9a7d-4225-860d-436b23adf0b2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 15:46:13.784617 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85f56bb-9a7d-4225-860d-436b23adf0b2 service placement] RBAC: Authorization granted {{(pid=80301) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 15:46:13.794976 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: [pid: 80301|app: 0|req: 1463/2901] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Jul 7 15:46:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 15:46:23.790693 dv-cinder-841302-5-iscsi devstack@keystone.service[80301]: DEBUG dbcounter [-] [80301] Writing DB stats keystone:SELECT=89 {{(pid=80301) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}